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Executive Summary 

State policymakers need evidence-based information to help them allocate scarce 
resources, compensate qualified staff, and determine the effectiveness of education spending. As 
part of its regional education laboratory work, the Southwest Educational Development 
Laboratory (SEDL) conducts research to inform policymakers and policy influencers from state 
legislatures, state departments of education, and governors’ offices in Arkansas, Louisiana, New 
Mexico, Oklahoma, and Texas. In this report, SEDL researchers investigate data collected and 
managed by state education agencies to determine whether new research can be conducted to 
support policy questions about education resources and student performance. 

Policy questions about education resources and student performance must be addressed to 
support local, state, and federal education priorities. Expectations for improved performance for 
all learners have been set by state accountability systems and the federal No Child Left Behind 
legislation. Increased attention on the resources needed to help students succeed, how these 
resources should be allocated, and whether spending and staffing strategies affect student 
performance bring to light the need to examine state data to inform these issues. We conclude 
that existing state education databases are a critical but underutilized data source that can inform 
and support policy decision making. 

We began this study with an assessment of data utilization in the four study states: 
Arkansas, Louisiana, New Mexico, and Texas. Lor this assessment we examined publicly 
available reports and summaries that used state education data in the four study states and 
addressed education resource issues. This assessment yielded very few rigorous studies that used 
existing state data to provide policy guidance on school resource questions. We conclude that 
researchers and policy audiences need (a) to expand the use of existing education data to support 
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decision making on instructional resources; (b) to examine data on both resources and student 
performance to better understand how education inputs and desired outputs relate; and (c) to 
incorporate data on student, school, and district characteristics when examining education 
resource issues. 

Before these important steps can be followed, however, policy researchers must have a 
clear understanding of the scope, quality, and availability of existing state data. This report 
contributes to this understanding by 

• describing in fine detail the data collected and housed by state education agencies in 
Arkansas, Louisiana, New Mexico, and Texas; 

• providing guidance to policy audiences and researchers about the questions that can be 
answered with these data, with discussion about using specific financial, staff, student 
performance, and student characteristic variables from each study state; and 

• discussing ways these data could be improved to expand the range of policy questions 
answered. 

Study Methodology 

For this study, we addressed the question “Do state databases allow the investigation of 
the relationship between fiscal and staff instructional resources and student performance?” In 
order to assess the capacity of existing state data to conduct such research, we (a) identified the 
key variables within fiscal and staff instructional resources, student performance, and student, 
school, and district characteristics needed to analyze critical policy questions; (b) developed 
criteria to assess the usability of these data; and (c) applied these criteria to identify resource 
allocation questions that can be answered with state data. We also examined whether 
commonalities exist in these data across the study states. 
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Can Existing State Data Be Used to Investigate Education Resources? 

SEDL researchers found that state data can and should be used for education policy 
research on instructional resource allocation. Specifically, the researchers concluded the 
following: 

1 . Dollars spent to support instruction can be examined using fiscal data broken down by 
function, object, and program categories. These expenditures also can be examined by accessing 
staff data that contain individual-level staff salaries. 

2. Individual staff salary data also can be linked with staff characteristics such as years of 
experience, highest degree, and certification information for more comprehensive analysis of 
staff compensation. 

3. Staff resources can be studied using state data, and full-time equivalency (FTE) counts 
on a wide range of staff categories are available in three of the four study states. Full-time 
equivalency counts or head counts can be matched with staff demographics or other 
characteristics or can be used to create staffing ratios such as pupikteacher or 
teacher:administrator. Actual class size information, however, is limited with the current data. 

4. Student performance data in each of the states are unique and have undergone changes 
in recent years. Although longitudinal analysis of student outcomes is limited by existing data in 
most states, student achievement scores can be matched with fiscal and/or staff resources at the 
school and district levels in all four study states. 

5. Student, school, and district characteristics are available in education databases in all 
four study states and are of critical value in understanding the relative influence of student, 
school, and district environments on resources and student performance. 
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What Can State Policymakers Do to Increase the Use and Quality of Existing State Education 
Data? 

Based on study findings, we recommend that policymakers, state data managers, and 
researchers work together to expand the use of state education data for resource allocation 
research. We also recommend that policymakers and state departments of education support and 
implement data improvements for greater applicability to informing state policy and find ways to 
make these data more available and accessible for research use. 

1 . Policy studies that help decision makers understand the relationship between resources 
and student performance are extremely rare in the four study states. Given what we learned from 
this study, policymakers and researchers need to work together to become more familiar with 
these data and use them to inform decisions. Increased use of existing state education data not 
only would provide increased information for policymakers but also would fuel a feedback 
mechanism for states to better understand how data need to be improved or expanded to serve 
information needs. 

2. Applying state education data to policy research purposes is a relatively recent priority 
and one that is not fully recognized by states, even today. In order for the data needs of policy 
research to move to the forefront, state policy audiences, data managers, and researchers must 
provide input on how existing data could be improved and changed for research purposes — in 
addition to more traditional reporting and monitoring purposes. 

3. When considering improvements and changes to state data, policymakers also must 
balance the time and resource burdens that changes in state data systems create for schools, 
districts, and state agencies. 
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4. As this study reveals, critical differences in the variable definitions used by the four 
study states and the range of information they collect leave researchers with few avenues for 
pursuing cross-state or regional studies on education resources. Policymakers, data managers, 
and researchers should maintain a dialogue with national data centers (such as the National 
Forum for Education Statistics) that attempt to bridge the gap between the unique needs of state 
data systems and the research benefits of establishing national data standards. 

5. We also recommend that policymakers and data managers consider the following 
targeted improvements to increase the usability of education data for resource allocation 
research. 

• Instructional expenditures at the school level are currently collected only in Texas; 
the other three study states currently collect this information at the district level. 
Adding school-level detail of how instructional resources are allocated would enable 
policy researchers to consider spending needs of schools in vaiying environments. 

• Teacher quality is quickly becoming one of the highest policy priorities due to the 
federal No Child Left Behind legislation and research results emphasizing the 
importance of good teachers as a predictor of student success. Data collected on 
teacher qualifications must align with federal priorities. We recommend that data 
managers improve the accuracy of teacher years of experience data and ensure that 
teacher certification data can be easily aligned to the teachers’ subject areas and grade 
levels. 

• Class size limits are imposed by both federal and state policy, and the benefits of 
smaller class sizes have been the topic of intensive study over recent years. In three of 
the four states, students are not linked to their classroom teachers, so a true estimate 
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of class size cannot be created. In New Mexico, where data do link students to 
teachers, access to these data is restricted from outside users. If policymakers are to 
fully understand the relationship between class size and student success, accurate 
measures, including data that link individual teachers to specific students or 
classrooms, must be created for use in policy analysis. 

• Professional development is currently unaccounted for in state education databases in 
the four study states. We recommend that states collect data on the amount and type 
of professional development that teachers receive and on the costs of investments in 
professional development to schools and districts. These data are essential to helping 
policymakers consider the costs and benefits of statewide initiatives to provide 
professional development to educators. 

6. If data are to be shared with outside users such as policy audiences and researchers, 
accessibility and availability are the critical first steps. We discuss the most important concerns 
regarding data access and recommend ways for policymakers and state education agencies to 
improve them. 

• Individual-level data are necessary to conduct in-depth analysis of student subgroups 
and relationships between different types of students, teachers, and resources. 
Policymakers and data managers should ensure that Family Educational Rights and 
Privacy Act (FERPA) regulations are interpreted in a consistent manner and should 
find ways for education agencies to share data from state databases while ensuring 
confidentiality of individuals. 

• Agencies that house state education data should ensure that procedures and staff are 
in place to assist data users. State education agency staff are a critical support to data 
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users for data requests, information about data structures and variables, and 
coordination with the multiple departments or related agencies that collect and 
manage education data. Clear procedures for data request should be established, and 
state data managers should communicate the time and cost needed to provide data. 

• The multiple departments at state education agencies that collect data should work 
toward creating centralized data systems that combine the multiple education 
databases that exist in each state. The Louisiana Educational Accountability Data 
System (LEADS) is an ongoing effort to implement an integrated data management 
system to support Louisiana’s education information needs. Also, the Texas Public 
Education Information Resource (TPEIR) database is being developed as a cross- 
agency data management system that combines primary, secondary, and higher 
education information. We recommend that state education agencies and 
policymakers in all states investigate similar initiatives to expand access to state 
education data. 

• Improvements need to be made so that data documentation is consistently available 
and comprehensive for all state education data. Information on variable definition, 
type, ranges (if applicable), and year-to-year changes should be made available. 
Agencies that manage education data should post updated documentation to agency 
Web sites to increase accessibility to this information. 

Based on this study of existing state education data, we conclude that there is a need for 
increased attention on and use of these data for policy research purposes. We also understand 
that much work remains to be done by policymakers, state data managers, and researchers to 
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create high-quality, user-friendly data that can be applied to important policy questions about 
education resources and student achievement. Such efforts would support the creation of 
evidenced-based information for policymakers and result in more effective decision making on 
the resources needed to help children succeed. 
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Chapter 1 
Introduction 

Information is one of the most important tools education decision makers need to help 
them effectively spend taxpayer money, allocate qualified staff, and determine the effectiveness 
of education investments. Decision makers must understand the role and influence of monetary 
and staff resources on the education system, and they must have information to help them decide 
where to invest limited resources for maximum effect on student learning. In this report, 
researchers from the Southwest Educational Development Laboratory (SEDL) describe the data 
collected and housed by state education agencies in Arkansas, Louisiana, New Mexico, and 
Texas. We provide guidance to researchers and state policy audiences, including policymakers 
and policy influences from state legislatures, state departments of education, and governors’ 
offices in Arkansas, Louisiana, New Mexico, Oklahoma, and Texas. We provide information 
about the questions that can be answered with existing data, and we discuss how these data can 
be improved to answer a wider range of questions. These data hold great promise for answering 
specific policy questions and should be utilized to their fullest potential to guide effective 
decision making. The conclusions and recommendations that result from this study will be of 
interest to policy audiences, data managers, and researchers from the four study states and 
nationally. 

Lederal priorities encourage policymakers to seek evidence-based information that 
combines sound research methods with reliable data. The quality and usability of relevant data, 
however, are unclear, especially with regard to existing state databases. In the recent past, states’ 
data needs were driven primarily by federal reporting requirements and state accountability 
priorities. Currently, policymakers are responding to a broader range of financial and 
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accountability issues than most state data systems were originally designed to track (Busch & 
Odden, 1997). Therefore, the need to scrutinize state data systems for both policymaking and 
research purposes has risen in priority. Past research on data use has noted an increase in the 
accuracy of data as a byproduct of its use, an incentive for both researchers and policymakers to 
turn to states for their data needs (Farland, 1997). 

As part of its mission, SEDL conducts research to inform education policymakers in its 
five-state region as part of its regional education laboratory work. In their 2003 policy study, 
SEDL researchers found that instructional resources are positively related to student performance 
and that strategies exist to allocate resources based on student learning needs. Specifically, we 
found that high-performing districts in Arkansas, Louisiana, New Mexico, and Texas allocated 
more fiscal and staff resources to instructional areas than did low-performing districts. We 
identified two critical strategies that support effective resource allocation: (a) data-driven 
decision making and (b) alignment of instructional goals with available resources (Pan, Rudo, 
Schneider, & Smith-Hansen, 2003). Researchers concluded that to improve resource allocation 
decisions, more information is needed about the relationship between specific instructional 
resources and student performance. 

We shared these findings with state policymakers at an annual state policy forum and 
during individual visits with policymakers in Arkansas, Louisiana, and Texas. As a result of 
these meetings we identified the following three areas of need: 

1 . Policymakers expressed substantial interest in study findings and a need for more 
information on how effective resource allocation can support student achievement. 

2. Policymakers explained that researchers’ use of federal databases (i.e., Common Core 
of Data) to determine resource allocation patterns in the study states weakened the impact of 
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results. In Texas, for example, state databases report expenditures for instruction across multiple 
functions depending on the purpose of the spending, whereas federal data aggregate spending in 
instruction into one function category in order to create a common standard for all states. 
Although federal databases provide reliable data that allow comparisons across states, 
researchers heard from state policymakers a clear need to focus on state-level data. State data, 
although of varying utility, offer more specific information on spending and staffing patterns 
within instructional areas than do federal data. State data sources also have increased currency 
and rely on measures of resources and performance that are typically more familiar to state 
audiences. 

3. Local and state decision makers understand that “one size does not fit all,” and they 
must consider how resource allocation strategies can be modified to fit local needs and 
environments. Further work is needed to identify accurate measures of local educational 
environments (e.g., student demographics, school and district characteristics) and to apply these 
measures to an examination of instructional resource allocation. 

The purpose of this study was to assess the capacity of existing state education databases 
to answer policy questions about instructional resource allocation and student performance. 
Existing state databases include organized collections of data managed by state entities for 
reporting, conducting research, and/or supporting policy and practice. Researchers at SEDL and 
nationally can use these findings to understand the feasibility and potential scope of using 
existing state data to conduct research on the allocation of instructional resources. Policymakers 
and practitioners, by better understanding the capacity of state data systems to conduct policy 
research, will be able to expand research about instructional resource allocation and student 
performance. 
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This report is divided into three major sections with detailed reference material in the 
appendixes. Chapter 2 describes how data on instructional resources and student performance 
have been utilized to support policy in Arkansas, Louisiana, New Mexico, and Texas. Chapter 3 
describes existing state education data in the four study states and discusses how researchers 
could use these data to answer policy questions related to instructional resources. We also 
explore what additional questions could be answered if data were improved. Chapter 4 poses 
recommendations for policymakers regarding how data might be better utilized and improved to 
support decision making. Findings on state data in New Mexico are represented in this report; 
however, these findings are limited and conjectural at times because data were not received from 
that state for analysis. Information about New Mexico data systems are based solely on printed 
documentation and interviews with state data managers. The methods used to conduct this study 
are explained in appendix A, and the remaining appendixes provide detailed descriptions of state 
education data in each of the four study states for the reader’s reference and use in planning 
future research studies. 
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Chapter 2 

Study Findings — Data Utilization in the Southwest Region 

Information on instructional resources and student achievement is critical for supporting 
education policy decisions. In order to better understand how data are used to support policy 
decisions in the southwestern region, SEDL researchers examined publicly available data reports 
and summaries, policy analyses, and research studies on resources and student performance that 
displayed or examined existing data in the four states. We identified reports and research studies 
that used state data on education resources through Web searches, discussions with state 
policymakers, and searches of relevant databases such as the Education Resources Information 
Center (ERIC). The following three areas of data use were targeted for this overview: 

1 . We sought to better understand which state data have been used to support policy 
decisions about education resource issues such as state spending levels or teacher resources such 
as teacher qualifications or compensation. 

2. We investigated whether state data have been used to better understand the 
relationship between resources and student performance. 

3. We asked whether previous research included data about student, school, or district 
characteristics to better interpret results. 

From this assessment of data utilization in the four study states, we conclude that state 
policymakers request and use existing data to understand education resources, especially 
regarding policy issues such as adequacy and equity of funding. We found research reports and 
data summaries on fiscal and staff resources in all four states. We also conclude, however, that 
state education data are not used to their full potential to inform policy issues regarding 
resources. Researchers have infrequently used existing data in the four study states to answer 
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research questions on the relationship between resources and student performance and the effect 
of student, school, or district characteristics. Data use has been largely limited to a few topical 
areas within the issue of education resources, such as school funding formulas, adequacy of 
education funding, and descriptive studies of finances and teachers. 

Education Resource Data and State Policy 

Education researchers and policy analysts have used state data in the four study states to 
answer questions related to education resources. In Arkansas, litigation prompted the legislature 
to commission a study of educational resource adequacy, and authors used existing financial and 
staffing data collected by the Arkansas Department of Education to provide cost estimates for 
providing adequate resources to schools and districts (Odden, Picus, & Fermanich, 2003). State 
financial data have been used in similar ways to support fiscal decision making in Louisiana and 
Texas. For example, state data in Louisiana were compiled for an analysis of education adequacy 
(Augenblick & Tetreault, 2001), and in Texas, state fiscal data were used to estimate a cost-of- 
education index for the state funding formula (Alexander, Gronberg, Jansen, Keller, Taylor, & 
Treisman, 2000). 

Researchers and state agencies also have compiled a wide range of descriptive 
information that could support policy decisions about instructional resources. Three of the four 
study states make report cards available to the public on the status of schools and districts; these 
report cards contain limited information about fiscal and/or staff resources. Additionally, the 
Louisiana State Department of Education (2003a) has examined teacher supply for the state and 
school districts. That agency has also produced data summaries on school staffing, revenues and 
expenditures, and salaries for use by the state board (Louisiana Department of Education, 

2003b). In New Mexico, limited descriptive data on teacher salaries, school funding, and 
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pupikteacher ratios from existing data are used in reports and policy analysis (Legislative 
Education Study Committee, 2002). A vast array of financial, staffing, and teacher data are 
available from the Texas Education Agency’s Web site, and information has been reported from 
state data on issues such as teacher mobility and retention (Texas Center for Educational 
Research, 1999; Texas Education Agency, 1994, 1995). 

Education Resources and Student Performance 

Although descriptive data that measure education resources do inform policy decisions, 
of specific interest to us is how policy analysts and researchers have used state data to explore 
the relationship between student performance and resource allocation. The reports cited above 
address resource issues; however, they do not link resource allocation to student performance. 
Evidence that explains this relationship and that also uses existing data from the four study states 
is limited. No major studies that examine the relationship between fiscal or staff resources and 
student performance using state data were found for Arkansas or New Mexico, and only a small 
number of studies were found that used Louisiana and Texas data. 

The Louisiana Department of Education recently published results of analyses of state 
data that examined teacher test scores, finding a relationship between teacher test scores and 
student performance. Also using Louisiana state data, Crone-Koshel and Singer (2002) compared 
the performance of schools receiving additional funds for implementing comprehensive school 
reform programs with Title I schools that did not have additional funds and found no significant 
differences. 

In Texas, several important studies have been conducted. Ferguson (1991) determined 
that a systematic relationship existed between school resource inputs and student outcomes using 
state data aggregated to the district level. Hanushek, Kain, and Rivkin (1998) and the Texas 
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Schools Project used state data to examine the relationship between teachers and student 
achievement. A resulting working paper suggested a relationship between teacher quality and 
student outcomes, with teacher quality accounting for at least 7.5 percent of the total variation in 
student achievement (Hanushek et ah, 1998). Another related study examined teacher salaries 
and found a weak but significant relationship between salaries of experienced teachers and 
student achievement (Hanushek, Kain, & Rivkin, 1999). These same researchers also examined 
teacher experience and salaries in schools with low-achieving students and found that 
experienced teachers are more likely to leave schools because of a disadvantaged, low-achieving 
student population than because of salary levels (Hanushek, Kain, & Rivkin, 2004). The Texas 
Legislature commissioned a study to determine the cost of achieving specified levels on student 
performance. The study examined resource inputs needed to educate students and also 
considered the different school environments that raise the level of necessary inputs. Authors 
found significant cost differences for students in different education environments (Gronberg, 
Jansen, Taylor, & Booker, 2004). 

Student, School, and District Characteristics 

Policymakers must consider how policy options might be different for schools or 
communities with different characteristics. Data about the needs of diverse learners could be 
applied to policy priorities such as narrowing the achievement gap and addressing the resource 
deficits faced by high-poverty and high-minority schools. As part of this assessment of data 
utilization, we examined how policy analysts and researchers have incorporated demographic 
data and other data on students, schools, and districts into investigations of education resources. 

SEDL’s 2003 resource allocation study found that after controlling for student poverty, 
student race/ethnicity, student special education status, and district size, significant differences in 
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resource allocation patterns between high-performing and low-performing districts disappeared 
(Pan et al., 2003). Other studies have confirmed the importance of considering the influence of 
demographic characteristics and other variables such as parent education and urban locale on the 
relationship between resource allocation and student achievement (Betts, Rueben, & Danenberg, 
2000; Ferguson & Ladd, 1996; Goldhaber & Brewer, 2000; Roza, Miles, & Foley, 2003). 

Researchers in the four study states have used state data to understand the role of 
demographic factors on student learning. In Louisiana, for example, a number of empirical 
studies have examined the role of poverty, race, ethnicity, rurality, and school composition on 
academic achievement (Caldas & Bankston, 1998; Franklin & Glascock, 1998; Kennedy, 1995; 
Lindsey & Fillippino, 2002). Arkansas researchers have studied the achievement gap between 
Black and White students and the effect of school size and poverty on academic achievement 
(Barnett, Ritter, & Lucas, 2003; Johnson, 2002; Mulvenon, Stegman, Ganley, & McKenzie, 
2002). Although these studies considered the role of demographic and other school 
characteristics in education achievement, they did not relate student achievement to education 
resources. Hanushek et al. (1999) incorporated demographic factors as control variables in their 
study of teacher resources and used student and family characteristics from the Texas Education 
Agency database, including race, ethnicity, gender, eligibility for free and reduced-price lunch, 
and student mobility. Harter (1999) also used data from the Texas Education Agency to examine 
the differences in resource allocation between high- and low-poverty elementary schools. She 
concluded that high-poverty schools are more susceptible to the ineffective use of resources. 

Based on this assessment of data utilization in the four study states, SEDL researchers 
conclude that state education data are underutilized and policy audiences need (a) to expand the 
use of existing education data to support decision making on instructional resources; (b) to 
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examine data on both resources and student performance to better understand how education 
inputs and expected outputs relate; and (c) to incorporate data on student, school, and district 
characteristics when examining education resource issues. To address the feasibility of utilizing 
existing state education data for policy research, we discuss the availability and usability of these 
data in the four states in chapter 3. We also consider the policy questions that can and cannot be 
addressed with existing education data. 
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Chapter 3 

Study Findings — Can Existing State Education Databases Be Used to Investigate 
Education Resources and Student Performance? 

The purpose of this study was to explore the capacity of existing state data to answer 
policy questions about the relationship between instructional resources and student performance. 
In this chapter we discuss how researchers can use state education data to answer policy 
questions about three resource areas: (a) instructional expenditures, (b) staff characteristics and 
teacher quality, and (c) instructional staffing patterns. We also describe student performance data 
as a desired outcome measure when examining the effectiveness of resource allocation. We 
describe student, school, and district characteristics data that researchers must consider due to the 
influence of socioeconomic and other factors on the availability of resources and the level of 
student outcomes. This analysis represents a regional overview of the four states. State-by-state 
descriptions of state education data in Arkansas, Louisiana, New Mexico, and Texas are 
provided in appendixes B-E. We expect that this information will guide researchers and 
policymakers to make effective use of existing data to answer policy questions. SEDL also 
expects to use this information to plan and conduct new research to support the improved 
allocation of resources in diverse state and local environments. 

SEDL researchers have particular interest in investigating policy issues in the 
southwestern region. Our focus for this study was to examine existing state data related to three 
broad areas of education resources: (a) the allocation of instructional dollars, (b) the importance 
of staff characteristics, and (c) the distribution of instructional staff. The allocation of 
instructional dollars addresses policy questions regarding the adequacy and equity of state 
funding formulas, compensation for teachers and administrators, and other fiscal considerations. 
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Policy questions about staff characteristics address how the quality of teachers and other staff 
relates to student success. Resource allocation research on the distribution of instructional staff 
investigates whether staff with different roles, qualifications, and demographics relate to student 
learning in different regions, districts, and schools. This chapter discusses the data available in 
state education databases that are relevant to each of these three resource areas. We provide an 
overview of overall data quality and discuss the questions that can be answered with existing 
state data as well as the questions that could be answered if data were improved. We also present 
the student outcome measures and demographic information available in the four study states. 

Instructional Expenditures 

Instructional expenditures are collected in state education databases and are generally 
defined as funds spent to support teaching and learning that occurs in the classroom, such as the 
cost of teachers, classroom materials and supplies, and contracted instructional services. 
Financial information can be used to examine spending patterns in instructional and other large 
expenditure categories such as general administration or support services (Pan et al., 2003). 

We investigated whether education data in the four study states could reveal more 
detailed information regarding how funds are spent within the category of instruction than 
federal data allow; we found state financial data to be of great potential benefit. As listed in 
Table 1, in two states, Arkansas and Louisiana, expenditures are organized into function 
categories that align to federal (Census F-33 form) categories. New Mexico and Texas divide 
instructional expenditures into two separate function categories. In all four states, financial 
databases break down instructional expenditures into object-level categories for each school 
district (see Table 1). Although the objects used in each state vary, these can be loosely 
categorized into salaries, benefits, contracted services, supplies and materials, and other 
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expenditures. Using these data, researchers can investigate whether different allocation patterns 
of object expenditures are related to district performance. Some states have more refined fiscal 
data. In Texas, instructional object expenditures can be investigated at the school level, and in 
Texas, Louisiana, and Arkansas, instructional expenditures are further divided into programmatic 
categories (regular education, special education, etc.). Also, as described in greater detail in 
appendixes B-E, states have additional subfunction and/or subobject categories that allow even 
more refined measurement of instructional spending. 



Table 1 

Description of the Instructional Expenditure Data Collected by the Four Study States by 
Function, Object, and Unit of Analysis 





Arkansas 3 


Louisiana 3 


New Mexico 


Texas 


Instruction-related 


• Instruction 


• Instruction 


• Direct 


• Instruction and 


function 


• Student support 


• Student 


instruction 


instruction-related 


categories 


• Instructional 


support 


• Instructional 


services 




staff services 


• Instructional 
staff services 


support 


• Instructional and school 
leadership 

• Support 
services — student 


Object categories 


• Salaries 


• Salaries 


• Personnel 


• Payroll costs 




• Benefits 


• Benefits 


services 


• Professional and 




• Professional 


• Professional 


• Employee 


contracted services 




purchased 


purchased 


benefits 


• Supplies and materials 




services 


services 


• Purchased 


• Other operating costs 




• Supplies and 


• Supplies and 


services 


• Debt service 




materials 


materials 


• Supplies and 


• Capital outlay — land. 




• Other objects 


• Other objects 


materials 

• Travel and 
training 

• Capital outlay 


buildings, and 
equipment 


Unit of analysis 


Program (for 
instruction only) 
District 


Program 

District 


District 


Program 

School 

District 



a Function and object categories align with federal functions and objects (Census form F-33). 
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Researchers also considered whether state education data could support an investigation 
of spending on individual staff salaries. In addition to the fiscal object category of salaries that is 
collected at the district and/or school level for all states, state education agencies collect 
information about the salary paid to each individual staff member (see Table 2). Also contained 
in staff databases are other characteristics of individual staff, such as the position a staff person 
holds (teacher, principal, etc.), education level, and demographic information. These data 
provide a rich source of information for researchers interested in analyzing how instructional 
staff salaries correspond to staff qualifications or other characteristics. 

The benefit of having individual-level data on staff salaries is the flexibility they afford 
researchers to conduct analyses on subgroups of staff, such as classroom teachers, principals, or 
aides. They also allow researchers to aggregate salary information to multiple levels (district, 
school, and grade in some states). Data on the amount spent for individual staff benefits are also 
available in Arkansas state databases. In New Mexico and in Arkansas prior to 2003, an 
individual’s contracted salary is recorded, but this amount cannot be linked with actual 
expenditure data recorded in the fiscal database. In Arkansas (since 2003), Louisiana, and Texas, 
individual staff salary information is recorded with financial function and object category labels 
so that this information aligns with aggregated salary information in the fiscal database. 
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Table 2 

Staff Salary Data Available in the Four Study States, Including Salary Measure, Unit of 
Analysis, Partial Salary Determination, and Benefits 



Salary Data 


Arkansas 


Louisiana 


New Mexico 


Texas 


Salary measure(s) 


• Total salary 


• Base salary 

• Additional 
compensation 
(3 types) 


• Base pay 


• Base pay 

• Supplemental 
pay 


Unit of analysis 


Individual 


Individual 


Individual 


Individual 


Can partial salaries be 
determined for part-time 
staff? 


Yes (since 2003) 


Yes 


Yes 


Yes 


Do salary data align with 
actual expenditures? 


Yes (since 2003) 


Yes 


No 


Yes 


Are expenditures for 
benefits available at the 
individual level? 


Yes 


No 


No 


No 



Data Quality. Data on instructional expenditures vary in quality from state to state. The 
information about each state’s data systems provides detailed state-by-state descriptions of the 
quality of these data and are located in appendixes B-E as follows: 

• Appendix B: Arkansas State Education Data 

• Appendix C: Louisiana State Education Data 

• Appendix D: New Mexico State Education Data 

• Appendix E: Texas State Education Data 

Overall, data that measure instructional spending and individual salaries have been 
consistent over time within each state. With the exception of changes to the Arkansas salary 
information, few substantive changes have been made in data variables during the study period 
(1999-2003). This consistency contributes to greater data reliability. State data managers 
identified few chronic problems in their financial data with regard to accuracy or completeness, 
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and districts have improved their data reporting systems over the years. 1 Other fiscal categories 
can be aligned only at the district level in all states except Texas, where fiscal data are reported 
at the school and program levels, and Arkansas and Louisiana, where district-level instructional 
expenditures are broken down into program categories. Salary data can be averaged across 
schools and districts for alignment with other staff data, student performance data, and other 
indicators. 

Spending on staff benefits is recorded at the individual level only in Arkansas, and 
benefit costs must be prorated across individual staff to estimate total compensation costs in the 
other three states. Benefits could be estimated in the remaining three states by prorating the total 
expenditures that a district spends on benefits for instructional staff. This district-level average 
would need to be applied universally across all staff included in the function category. In most 
states, this minimally includes teachers, substitutes, and teacher aides, further decreasing the 
possibility of extracting a usable measure of benefits for any individual staff. Also, information 
about bonuses or other incentives for teachers in high-demand subject areas or geographic 
regions are not separated from base salaries except in Louisiana. 

Questions That Can Be Answered. As shown in Table 3, policy research questions about 
instructional spending can be answered with existing state education data. State fiscal databases 
contain expenditure function categories that are familiar to state and local policy audiences, and 
researchers can examine how instructional dollars are allocated in districts of varying levels of 
student performance. Student performance scores are necessary for this analysis, and these data 
are available in each of the four study states, as discussed in greater detail beginning on page 38. 
Each state compiles test scores on criterion-referenced and/or norm-referenced exams for 

1 The identification of other important sources of error in these data such as inconsistency in 
coding data at the district level were beyond the scope of this study. 
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students in specific grades and subject areas. These test results can be requested from the state 
for use in resource allocation research. By merging student performance data and fiscal data, 
researchers can investigate which object-level expenditure categories are more likely to affect 
student performance. For these analyses, researchers can examine district-level spending in all 
four states and school-level spending in Texas. Researchers also can compare function- level 
expenditures to determine if districts of varying levels of performance allocate instructional and 
administrative dollars differently. 
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Table 3 

Policy Questions About Instructional Resources That Can be Answered With Existing Data in 
the Four Study States, Including Relevant Variables and Level of Alignment 



A. Do higher performing districts allocate more instructional dollars to salaries and 
benefits? 




Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant 

variables 


• Expenditure objects 
within instruction 
function 

• Student 

performance scores 


• Expenditure objects 
within instruction 
function 

• Student 

performance scores 


• Expenditure objects 
within direct 
instruction function 

• Student 

performance scores 


• Expenditure objects 
within instruction 
and instruction- 
related services 
function 

• Student 

performance scores 


Level at 
which 


District 


District 


District 


School 

District 



data can 
be 

aligned 

Notes Salaries and benefits constitute a large proportion of object expenditures in instruction, 
decreasing potential for identifying variation in spending between object categories. 



B. How do districts of varying levels of performance allocate administrative vs. instructional 
dollars? 





Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant 


• Expenditure 


• Expenditure 


• Expenditure 


• Expenditure 


variables 


functions: 


functions: 


functions: direct 


functions: 




instruction, student 


instruction, student 


instruction, 


instruction and 




support, 


support, 


instructional 


instruction-related 




instructional staff 


instructional staff 


support, 


services, 




services, general 


services, general 


administration 


instructional and 




administration, 


administration, 


• Student 


school leadership, 




school 

administration 
• Student 

performance scores 


school 

administration 
• Student 

performance scores 


performance scores 


student support 
services, 
administrative 
support services 
• Student 

performance scores 


Level at 
which 
data can 
be 

aligned 


District 


District 


District 


School 

District 
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Individual-level salary information provides an even greater potential for research and 
policy and can be applied to important questions. Table 4 displays the relevant variables 
available in the four states to address such questions. Existing data can investigate the 
relationship between teacher pay and such desired outcomes as increased student performance or 
the retention of qualified teachers. These data also can examine whether salaries are distributed 
equitably between schools and districts. State education data in all four study states allow 
researchers to investigate the relationship between teacher salaries and student performance at 
the district and school levels. Grade-level analysis could be conducted with Louisiana, New 
Mexico, and Texas salary and student performance data. Individual teacher- and student-level 
analysis could be conducted with New Mexico data. 

Salary information also can be linked to teacher experience and mobility in order to 
examine the effect of pay on retention. Teacher mobility is not a standard variable in existing 
databases but can be calculated using information on the school and district assignment for 
teachers over multiple years. Those teachers who change schools or districts over time, or who 
leave the system altogether, can be recorded as mobile teachers; those who do not change can be 
recorded as retained teachers. Teacher salary, teacher experience, and school and district 
identifier data are collected on an individual level, so researchers can examine the relationship 
between salary and teacher retention at the individual, school, or district level. Teacher 
qualification data are collected in all four study states as well. These data can help researchers 
consider whether the relationship between salary and retention changes for teachers of varying 
education levels, years of experience, or certification status. 

The distribution of salaries for teachers and other staff across schools and districts also 
can be examined using existing data. Distribution of other instructional expenditures (benefits, 
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supplies and materials, contracted services) can be examined at the district level for all states and 
at the school level in Texas. Further, these data can be grouped using characteristics such as 
high- or low-performing school districts to arrive at comparative information on instructional 
spending. 
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Table 4 

Policy Questions About Salaries That Can be Answered With Existing Data in the Four Study 
States, Including Relevant Variables and Level of Alignment 



C. How is teacher pay linked to student performance? 





Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Total pay for 


• Base pay or total 


• Base pay for 


• Base pay for 




teachers 


pay for teachers 


teachers 


teachers 




• Benefits for 


• Student 


• Student 


• Student 




teachers 3 


performance 


performance 


performance 




• Student 
performance 
scores 


scores 


scores 


scores 


Level at 


School 


Grade 


Individual 


Grade 


which data can be 


District 


School 


Grade 


School 


aligned 




District 


School 

District 


District 



Notes a 2002-2003 data 

only 



D. What is the effect of salary on retaining qualified teachers? 



Relevant variables 


• Total pay for 


• Base pay or total 


• Base pay for 


• Base pay for 




teachers 


pay for teachers 


teachers 


teachers 




• Years of 


• Years of 


• Years of 


• Years of 




experience 


experience 


experience 


experience 




• Mobility of 


• Mobility of 


• Mobility of 


• Mobility of 




teachers 


teachers 


teachers 


teachers 




(calculated) 


(calculated) 


(calculated) 


(calculated) 




• Certification 


• Certification 


• Certification 


• Certification 




• Education level 


• Education level 


• Education level 


• Education level 


Level at which 


Individual 


Individual 


Individual 


Individual 


data can be aligned 


School 


School 


School 


School 




District 


District 


District 


District 


Notes 






Certification 
information is 
limited due to lack 
of issue dates. 




E. Are teacher salaries distributed equitably between schools and districts? 


Relevant variables 


Total pay for 


Base pay for 


Base pay for 


Base pay for 




teachers 


teachers 


teachers 


teachers 


Level at which 


School 


School 


School 


School 


data can be aligned 


District 


District 


District 


District 



Notes Data from non-education databases could also be used to support this analysis, such 

as cost-of-living indicators. 
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Questions That Could be Answered With Additional Data. Additional data that would 
enable policy researchers to address a wider range of questions regarding instructional spending 
include better measures for total compensation and professional development. As mentioned 
above, salary information is available for individual staff persons; however, data on dollars spent 
on benefits are available at the individual staff level in only one state. If all states were to collect 
individual-level data on the cost of benefits, bonuses, and incentives, researchers could calculate 
a more accurate estimate of total staff compensation. These more refined data also could be used 
to examine the influence of benefits and incentives on teacher recruitment or the successful use 
of incentives for hiring and retaining teachers in shortage areas (bilingual education, special 
education, etc.). 

Professional development is another area of instructional spending and a policy issue 
about which decision makers need information. No fiscal spending category or individual staff 
data elements currently exist to help policy researchers understand how investments in 
professional development for teaching staff improve student outcomes. Researchers need actual 
or dollar-equivalent measures for teacher time, stipends, travel expenses, and costs for teacher 
substitutes related to professional development that can be prorated or directly assigned to 
individual teachers. The No Child Left Behind legislation emphasizes the importance of tracking 
the number of hours a teacher spends in professional development. Information on the content of 
professional development for teachers is also needed. Researchers could use these data to address 
questions about the effectiveness of professional development, its relative costs, and the 
distribution of professional development resources across schools and districts. 
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Staff Characteristics Data and Teacher Quality 

Data on staff characteristics are important for policy audiences who need to understand 
the contribution of instructional and administrative staff in the learning process. A major 
emphasis within staff-related policy concerns is the qualifications of teachers. A number of 
critical policy questions need answers so that states can make research-based decisions to 
improve teacher quality. Questions that can be investigated using existing state data include 
those related to teacher experience, education, and certification. 2 

As displayed in Table 5, teacher characteristics data available across the four study states 
are relatively similar. For each of the states, researchers can measure basic demographics of 
teachers, educational attainment, years of experience, and scores on state teacher tests. Each of 
the four states also collects, minimally, data on teacher certification type, grade level, and subject 
area. All states except New Mexico also collect the issue date and expiration date of the 
certification, and Arkansas and Louisiana track national board certification. Arkansas, Louisiana, 
and Texas record teachers’ route to certification (traditional or alternative). All states except 
Arkansas provide data to measure the proportion of a full-time equivalency (FTE) position a 
teacher holds. All four study states collect data on individual teachers, and these data can be 
linked to school and district identifiers for analysis at these aggregated levels. In New Mexico, 
teachers are linked to students through class codes and identification numbers. 3 



2 These areas represent some important and easily quantifiable indicators of teacher quality. 
However, it is important to note that other measures of quality such as application of pedagogical 
techniques, teacher motivation, and classroom management skills are not collected via state 
databases. 

3 Louisiana began linking individual students and teachers beginning in 2004-2005; however, 
these data are beyond the time period of this study and were not examined for this report. 
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Table 5 

Teacher Characteristics Data Available in State Databases in the Four Study States, Including 
Characteristics Variables, Certification Data, and Level of Alignment 





Arkansas 


Louisiana 


New Mexico 


Texas 


Teacher 


• Gender 


• Gender 


• Gender 


• Gender 


characteristics 


• Race/ethnicity 


• Race/ethnicity 


• Race/ethnicity 


• Race/ethnicity 




• Birth date 


• Full-time 


• Birth date 


• Birth date 




• Years of 


equivalency of 


• Full-time 


• Full-time 




experience 


position 


equivalency of 


equivalency of 




• Lowest and 


• Years of 


position 


position 




highest degree 


experience 


• Years of 


• Years of 




• Highest degree 


• Highest degree 


experience 


experience 




institution 


• Degree 


• Highest degree 


• Highest degree 




• Score on teacher 


institution 


• Bachelor’s degree 


• Bachelor’s 




tests 


• Score on teacher 
tests 


and highest 
institution 
• Score on teacher 
tests 


degree institution 
• Score on teacher 
tests 


Teacher 


• Type 


• Type 


• Type 


• Type 


certification 


• Grade level 


• Grade level 


• Grade level 


• Grade level 


data 


• Subject area 

• Effective date 

• National board 

• Route to 
certification 


• Subject area 

• Effective date 

• National board 

• Route to 
certification 


• Subject area 


• Subject area 

• Effective date 

• Route to 
certification 


Level at which 


Individual 


Individual 


Individual 


Individual 


data are 


School 


School 


School 


School 


available or can 
be aligned 


District 


District 


District 


District 



Data Quality. Data measuring staff characteristics are, for the most part, of sufficient 
quality to be used for policy research purposes. Data on teachers are more extensive than other 
staff data and include individual-level information on experience, education levels, certification 
status, and teacher test scores. Teacher data are relatively accessible by special request to state 
education agencies because individual staff data are protected with a lower level of 
confidentiality than individual student data. 4 A number of challenges related to teacher data, 

4 For those states that use Social Security numbers as unique identifiers, these numbers must be 
stripped or scrambled by the education agency before release to the outside users. 
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however, limit the usability of these data to answer policy questions. According to interviews 
with data managers across the four states, teacher experience data are of limited reliability due to 
reporting error at the level of local collection. Also, teacher certification information is 
problematic for three reasons: 

1 . Teacher certification data are collected and managed by a separate department from 
that which collects and manages other teacher data, increasing the probability of misalignment 
when merging these two sets of data. This problem is most apparent in states where different 
agencies use different unique identifiers or must scramble unique identifiers for outside data 
users. 

2. Teacher certification data are collected in databases that are cumulatively updated 
without year-to-year archive information on teachers. 

3. Certification requirements and teacher tests change frequently, making longitudinal 
analysis of certified teachers extremely difficult. 

Policy Questions That Can Be Answered. As presented in Table 6, existing data on 
teachers can answer policy questions about teacher education, years of experience, certification, 
and teacher test scores. Teacher information is used in standard reporting by state education 
agencies, and teacher years of experience and education data are used to determine teacher salary 
levels. All four study states collect the total number of years a teacher has been in the profession, 
which can be used to examine the relationship between teacher experience and student 
achievement. The quality of these data varies from state to state and is dependent on unverified 
self-reports from either individual teachers or school districts. As summarized in Table 6, these 
data can be used to explore the relationship between teacher experience and student achievement. 
In order to address the relative unreliability of the self-reported experience information, 
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researchers may choose to crosscheck these data across a span of years. Since teachers and 
students are not aligned in state databases, except in New Mexico, researchers need to aggregate 
data to school or district levels in the remaining three states. 

Researchers also can use state data to address the relationship between teacher education 
and student achievement (see Table 6). Teacher education variables in the four study states 
include highest degree and degree institution. Degree data usually include bachelor’s, master’s, 
and doctoral levels. Degree institution data are limited because in all four states only the names 
of in-state institutions attended are collected, and states use a universal label for out-of-state 
degrees. 

Teacher certification is another important area of policy research that can be supported 
with state data, as shown in Table 6. Certification data in the four states are compiled in a 
separate database from other staff data and are managed by a department or agency whose prime 
responsibility is overseeing the licensure or certification of teachers. Measures available for 
teacher certification in each of the four study states include the certification type, subject area, 
and grade level. Arkansas, Louisiana, and Texas also track teachers’ route to certification, and 
Texas provides an estimate of the number of teachers in each school who are teaching in their 
field of certification. This estimate of in-field teachers, however, is imprecise because 
elementary teachers do not teach specific subject areas and secondary teachers potentially teach 
multiple subjects. Also, the in-field determination does not consider teachers’ degree major, 
which is not currently collected by any of the four study states. Arkansas and Louisiana also 
collect data on whether a teacher is national board certified. It is important to note that teacher 
certification data in New Mexico are limited with regard to their usability for policy research. As 
described above, teacher certification databases are organized in a cumulative fashion, so the 
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issue date and expiration date of certifications are essential for conducting analysis on these data. 
In New Mexico, the certification database lacks dates when certifications were granted or 
expired, making it impossible to identify whether or not a teacher was certified during any given 
study period. 

Scores on teacher tests are also available in state databases for teachers in all study states. 
These data represent an array of test scores because teachers have had the option to take one of a 
number of tests or required tests have changed from year to year. Researchers can conduct 
analysis of subsets of teachers who took the same exam or explore the possibility of 
standardizing scores across different exams. Teachers in some states have no test score 
information if they were certified before testing requirements were instituted. Also, since 
teachers are assessed on a pass/fail basis and test takers may not be motivated to score as high as 
possible, test scores may reflect an inaccurate level of teachers’ knowledge. 
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Table 6 

Policy Questions About Teachers That Can be Answered With Existing Data in the Four Study 
States, Including Relevant Variables and Level of Alignment 



F. What is the relationship between teacher experience and student achievement? 





Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Years of 
experience in the 
profession 

• Student 
performance 
scores 


• Years of 
experience in the 
profession 

• Student 
performance 
scores 


• Years of 
experience in the 
profession 

• Student 
performance 
scores 


• Years of 
experience in the 
profession 

• Student 
performance 
scores 


Level at which 
data can be aligned 


School 

District 


School 

District 


Individual 

School 

District 


School 

District 


Notes 


Experience data are not fully reliable due to self-reporting error. Researchers may 
choose to crosscheck these data across a span of years. 


G. What is the relationship between teacher education and student achievement? 




Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Highest degree 

• Degree 
institution 

• Student 
performance 
scores 


• Highest degree 

• Degree 
institution 

• Student 
performance 
scores 


• Highest degree 

• Degree 
institution 

• Student 
performance 
scores 


• Highest degree 

• Degree 
institution 

• Student 
performance 
scores 


Level at which 
data can be aligned 


School 

District 


School 

District 


Individual 

School 

District 


School 

District 


Notes 


• Degree major not available 

• Out-of-state degree institutions are not specified 
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H. What is the relationship between teacher certification and student achievement? 





Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Certification 


• Certification 


• Certification 


• Certification 




type, subject 


type, subject 


type, subject 


type, subject 




area, and grade 


area, and grade 


area, and grade 


area, and grade 




level 


level 


level 


level 




• National board 


• National board 


• Student 


• Route to 




certification 


certification 


performance 


certification 




• Route to 


• Route to 


scores 


• In-field teacher 




certification 


certification 




• Student 




• Student 


• Student 




performance 




performance 


performance 




scores 




scores 


scores 






Level at which 


School 


School 


School 


School 


data can be aligned 


District 


District 


District 


District 


Notes 






Certification status 


In-field teacher 








data are 


data are estimated 








problematic due to 


and may not be a 








lack of issue dates. 


reliable measure. 


I. What is the relationship between teacher test scores and student achievement? 




Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Teacher test 


• Teacher test 


• Teacher test 


• Teacher test 




score 


score 


score 


score 




• Student 


• Student 


• Student 


• Student 




performance 


performance 


performance 


performance 




score 


score 


score 


score 


Level at which 


School 


School 


School 


School 


data can be aligned 


District 


District 


District 


District 


Notes 


Teacher test score data are an inconsistent measure due to variability in tests over 



time and the pass/fail nature of the tests. This question can be answered using data 
on subsets of teachers, which may bias results. 



Data on teacher experience, education, and certification in the four study states can be 
merged with other variables to answer additional policy questions (see Table 7). Researchers 
can, for example, merge teacher characteristics and teacher salary information at the individual 
staff level in all four states and ask whether higher teacher salaries buy teachers with more 
experience, higher education levels, and advanced certification status. Researchers also can use 
teacher data to examine the distribution of teachers who were educated at different teacher 
education institutions or to determine whether rural areas have a higher rate of uncertified 
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teachers. Arkansas, Louisiana, and Texas data can inform studies regarding the relationship 
between teacher retention and route to certification. 



Table 7 

Additional Policy Questions About Teachers That Can be Answered With Existing Data in the 
Four Study States, Including Relevant Variables and Level of Alignment 



J. Do higher teacher salaries buy teachers with more experience, higher education levels, 
and advanced certification status? 





Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Total pay for 


• Base pay for 


• Base pay for 


• Base pay for 




teachers 


teachers 


teachers 


teachers 




• Years of 


• Years of 


• Years of 


• Years of 




experience in the 
profession 

• Highest degree 

• Certification 


experience in the 
profession 

• Highest degree 

• Certification 


experience in the 
profession 

• Highest degree 

• Certification 


experience in the 
profession 

• Highest degree 

• Certification 




type, subject 
area, and grade 
level 


type, subject 
area, and grade 
level 


type, subject 
area, and grade 
level 


type, subject 
area, and grade 
level 




• National board 


• National board 


• Student 


• Student 




certification 


certification 


performance 


performance 




• Student 


• Student 


scores 


scores 




performance 


performance 








scores 


scores 






Level at which 


School 


School 


Individual 


School 


data can be aligned 


District 


District 


School 

District 


District 


Notes 






Certification status 
data are 

problematic due to 
lack of issue dates. 






Experience data are not fully reliable due to self-reporting error. Researchers may 




choose to crosscheck these data across a span of years. 
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K. What is the pattern of distribution of teachers who were educated at different teacher 
education institutions? 




Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Degree 
institution 

• School code 

• District code 


• Degree 
institution 

• School code 

• District code 


• Degree 
institution 

• School code 

• District code 


• Degree 
institution 

• School code 

• District code 


Level at which 
data can be aligned 


School 

District 


School 

District 


School 

District 


School 

District 


Notes 


Data do not designate specific out-of-state degree institutions but use a generic code 
for all such institutions. 


L. Do rural areas have a higher rate of uncertified teachers? 




Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Certification 
type, subject 
area, and grade 
level 

• National board 
certification 

• Locale code 


• Certification 
type, subject 
area, and grade 
level 

• National board 
certification 

• Locale code 


• Certification 
type, subject 
area, and grade 
level 

• Locale code 


• Certification 
type, subject 
area, and grade 
level 

• Locale code 


Level at which 
data can be aligned 


School 

District 


School 

District 


School 

District 


School 

District 


Notes 






Certification status 





data are 

problematic due to 

lack of issue dates. 

Locale codes for schools and districts are located in federal Common Core of Data. 



M. What is the relationship between teacher retention and route to certification? 





Arkansas 


Louisiana New Mexico 


Texas 


Relevant variables 


• School code 
(multiple years) 

• District code 
(multiple years) 

• Route to 
certification 


• School code • N/A 

(multiple years) 

• District code 
(multiple years) 

• Route to 
certification 


• School code 
(multiple years) 

• District code 
(multiple years) 

• Route to 
certification 


Level at which 
data can be aligned 


School 

District 


School N/A 

District 


School 

District 


Notes 


Teacher mobility/retention can be inferred by comparing teachers’ school and 
district assignment codes over multiple years. 

During the study period New Mexico did not have an alternative route to 
certification. 
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Questions That Could be Answered With Additional Data. Improved and expanded 
collection of teacher characteristics would improve policy research on teacher issues. For 
example, although teacher experience is measured in all four study states, these states could 
improve the reliability of the measures for research use by helping districts better understand 
reporting definitions of this variable — especially for teachers who transfer between districts. 
Also, none of the four study states collect information to estimate accurately whether a teacher is 
teaching in-field based on his or her degree major. This measure is a critical component of 
“highly qualified teachers” as defined by the No Child Left Behind Act. If states would collect 
data on teachers’ degree major, researchers could align these data with the subject area the 
teacher is certified to teach and the subject the teacher actually teaches to estimate the number, 
distribution, and effectiveness of in-field teachers. 

Teacher retention and mobility are important measures that researchers currently can 
calculate using teacher school and district assignment codes over time. If states included a 
teacher mobility measure in state education databases, this would provide a consistent measure 
for researchers and would reduce error and variability in the way outside data users calculate 
mobility. Teacher test score data are limited for research purposes and must be used with caution 
for reasons beyond data collection and management. These data reflect multiple teacher tests that 
have changed over time, making comparisons between teachers difficult except for subsets of 
teachers and for specific years. The pass/fail scoring structure also reduces the accuracy of 
teacher test scores as a measure of teachers’ knowledge. This measure could be improved by 
creating a universal standard for teacher tests and by adding incentives for teachers to score as 
high as possible on the tests. 
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Instructional Staffing Patterns 

As discussed above, policy questions about teacher resources can be investigated using 
existing state education databases. As presented in Table 8, staff counts and ratios are collected 
by the four study states and can be used to inform policy. The states also collect detailed 
information on all classified and certified staff, including salaries, position, years of experience, 
gender, and race/ethnicity. Other characteristics that are collected on all staff in one or more of 
the four study states include educational attainment and degree institution. Staff databases in 
Louisiana, New Mexico, and Texas also include information about the full-time equivalency 
(FTE) of staff positions, allowing researchers to determine staff allocations with more precision 
than staff head counts permit. In Arkansas, Louisiana, and Texas, for individuals with multiple 
job positions, researchers can determine what proportion of an individual salary compensates 
each position. Individual staff records also contain school assignment, making it possible for 
researchers to build staff profiles for a specific school or subgroup of schools. School-level staff 
ratios can be calculated or combined with student enrollment data to create student and staff 
ratios (pupikteacher, pupikadministrator, teacher: administrator, teacher:aide, etc.). 

Class size is an important data variable for conducting policy research. Policy audiences 
need to understand the relative cost of reducing class size and its relationship to student 
performance. Class size information can be estimated in a variety of ways with varying accuracy. 
A calculation of the ratio between the number of students in a school and the number of teachers 
in the school is the least accurate measure of class size due to (a) the probability that not all 
teachers in a school are regular classroom teachers, (b) the fact that in most secondary schools 
teachers are assigned to multiple classes, and (c) the possibility that team teaching or other 
nontraditional classroom structures exist. As shown in Table 8, in Arkansas and Texas existing 
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data allow researchers to calculate pupihteacher ratios (Texas also calculates a pupilxlass ratio 
in secondary schools). However, in Arkansas this ratio must be calculated with a head count of 
teachers, rather than the more refined full-time equivalency (FTE) count of teachers. A more 
accurate measure of class size is available in Louisiana, where local school districts track and 
report actual class sizes to the state. In New Mexico, the databases are structured so that 
researchers, theoretically, can link teachers to the students they teach in specific classes to 
calculate a relatively accurate class size measure . 5 



5 As of this writing SEDL researchers could not test whether class size could actually be 
computed due to unavailability of data from the New Mexico Public Education Department. 
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Table 8 

Information on Classified and Certified Staff Available in State Databases for the Four Study 
States, Including Counts, Staff Categories, Characteristics, Class Size Estimates, and Ratios 





Arkansas 


Louisiana 


New Mexico 


Texas 


Staff counts 


• Head count 


• Head count 

• Full-time 
equivalency (FTE) 
by position 


• Head count 

• Full-time 
equivalency (FTE) 
by position 


• Head count 

• Full-time 
equivalency (FTE) 
by position 


Type of staff 
included 


• Classified and 
certified staff 


• All staff 


• Classified and 
certified staff 


• Classified and 
certified staff 


Staff 

characteristics 


• Position 

• Years of 
experience 

• Education level 

• Degree institution 

• Gender 

• Race/ethnicity 

• Birth date 


• Position 

• Years of 
experience 

• Education level 

• Gender 

• Race/ethnicity 


• Position 

• Years of 
experience 

• Education level 

• Degree institution 

• Gender 

• Race/ethnicity 

• Birth date 


• Position 

• Years of experience 

• Education level 

• Gender 

• Race/ethnicity 


Class size 
estimate 


Pupikteacher ratio 
can be calculated 
using teacher head 
count, not full-time 
equivalency count. 


Class size ranges are 
reported at the 
school level. 


Potential to compute 
class size using 
individual-level 
student, teacher, and 
class data 


Pupikteacher ratio 
and pupikclass ratio 
(secondaiy) are 
available in state 
data. 


Staff ratios 


Can be calculated 
(using head count 
not full-time 
equivalency 
count): 

• Pupikteacher 

• Pupikadministrator 

• Teacher: adminis- 
trator 


Can be calculated: 

• Pupikteacher 

• Pupikadministrator 

• Teacher: adminis- 
trator 


Can be calculated: 

• Pupikteacher 

• Pupikadministrator 

• Teacher: adminis- 
trator 


Can be calculated: 

• Pupikteacher 

• Pupikadministrator 

• Teacher: adminis- 
trator 



Data Quality. Staff data in all four study states are collected at the individual level and 
are of sufficient quality for research use. Individual-level data can be requested from state 
education agencies, and aggregated reports of these data are available from state agency Web 
sites. Data collection procedures for staff data have been in place for many years and have not 
changed substantially during the study period (1999-2003) except in Arkansas. The consistency 
and relatively clear and detailed reporting instructions for districts contribute to the quality of 
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staff data. Since data are collected for individual staff and relevant school or district labels are 
attached to these data, they provide great potential for aligning to student performance data, 
student characteristics, and school and district characteristics. Researchers also must realize the 
complexity of staff data. Understanding, for example, how teachers with multiple roles and 
responsibilities are recorded in each state’s databases requires careful study of the data 
documentation and/or conversations with state data managers. 

Questions That Can be Answered. The four study states all collect individual-level staff 
data that provide flexibility for policy research about staffing patterns. Using staff full-time 
equivalency (FTE) counts, researchers can estimate staff resources by position at every school. 
Combined with student performance data, researchers can investigate how staff resources differ 
across schools with varying student performance levels (see Table 9). Such analyses can be 
conducted at the school or district level. Researchers also can use demographic data that are 
collected in each of the four study states (for a description of available demographic data, see the 
discussion beginning on page 38) and staff characteristics, such as years of experience or 
race/ethnicity, to better understand the distribution of staff. For example, researchers can 
examine the distribution of teachers and administrators with varying years of experience in 
small, medium, and large schools. 

Researchers also can use pupihstaff ratios to understand whether allocating more 
administrators, teachers, or aides is related to student performance (see Table 9). For example, all 
four study states collect data to study how class size relates to student performance. As discussed 
above, estimates for class size would vary from pupihstaff ratios to actual reported class sizes, 
depending on the data available in each state. 
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Table 9 

Policy Questions About Staff Resources That Can be Answered Using State Data in the Four 
Study States, Including Relevant Variables and Level of Alignment 



N. How do staff resources differ across schools with varying student performance levels? 





Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Staff head count 


• Staff full-time 


• Staff full-time 


• Staff full-time 




• Staff position 


equivalency 


equivalency 


equivalency 




code 


(FTE) count 


(FTE) count 


(FTE) count 




• Student 


• Staff position 


• Staff position 


• Staff position 




performance 


code 


code 


code 




scores 


• Student 


• Student 


• Student 






performance 


performance 


performance 






scores 


scores 


scores 


Level at which data 


School 


School 


School 


School 


can be aligned 


District 


District 


District 


District 


Notes 


FTE data are not 










available. 









O. What is the distribution of teachers and administrators with varying years of experience 
in small, medium, and large schools? 





Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Teacher head 


• Teacher full-time 


• Teacher full-time 


• Teacher full-time 




count 


equivalency 


equivalency 


equivalency 




• Administrator 


(FTE) count 


(FTE) count 


(FTE) count 




head count 


• Administrator 


• Administrator 


• Administrator 




• Years of 


full-time 


full-time 


full-time 




experience 


equivalency 


equivalency 


equivalency 




• School 


(FTE) count 


(FTE) count 


(FTE) count 




enrollment 


• Years of 


• Years of 


• Years of 






experience 


experience 


experience 






• School 


• School 


• School 






enrollment 


enrollment 


enrollment 


Level at which data 


School 


School 


School 


School 


can be aligned 
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P. What is the relationship between class size and student performance levels? 




Arkansas 


Louisiana 


New Mexico 


Texas 


Relevant variables 


• Teacher head 
count 

• Enrollment 

• Student 
performance 
scores 


• Class size range 

• Student 
performance 
scores 


• Student, teacher, 
and class ID 

• Student 
performance 
scores 


• Pupihteacher and 
pupikclass ratios 

• Student 
performance 
scores 


Level at which 


School 


School 


Class 


School 


data can be aligned 


District 


District 


School 

District 


District 


Notes 


• Class size is not 
estimated in state 
data 

• Pupihteacher 
ratio can be 
calculated 


• Class size 
ranges: 1-20, 
21-26,27-34, 
more than 34 


• Class database 
aligns student, 
teacher, and class 
ID that can be 
aggregated to 
estimate class 
size 


• Pupihteacher 
(elementary) and 
pupihclass 
(secondaiy) 
ratios are in state 
data 



Questions That Could be Answered With Additional Data. The relationship between class 
size and student performance is of interest at both the state and federal levels of education 
decision making. If data were collected that link students to teachers and teachers to specific 
classes in all four study states, this policy issue could be investigated more accurately. 

Student Performance Data and Student, School, and District Characteristics 

Student performance data and demographic data on students, schools, and districts are 
critical elements for conducting research on resource allocation and student performance. 

Student performance data provide important measures of school and district success. Policy 
analysis that includes student achievement data can uncover resource strategies and inputs that 
relate to improved student performance. Demographic information is useful for two reasons. 
First, demographic characteristics such as student poverty and minority status are known to 
influence student performance measures and must be accounted for when investigating the 
relationship between resources and student performance. Second, demographic characteristics 
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can help researchers identify — and policymakers understand — specific resource needs, 
circumstances, and barriers faced by schools and districts with varying wealth, geographic 
characteristics, and student populations. 

Student performance data described in this report include only scores on student 
achievement tests compiled and managed by state education agencies in each of the four study 
states (see Table 10). Other measures of student performance such as attendance and graduation 
rates are not discussed in this report. Student achievement testing generally reflects instructional 
goals and priorities established by state policymakers and thus vary from state to state. States use 
norm-referenced exams, criterion-referenced exams, or both for student accountability purposes. 
Standardized test scores are available for varying grades and subject areas, depending on state 
requirements. All states have tests for core subject areas (math, reading or English language arts) 
and other subjects such as science and social studies. In all four states, tests have changed over 
the years and grades tested have changed, making longitudinal comparisons difficult over long 
time spans. Student test scores are recorded for each individual student tested. Additionally, 
basic demographic information about the student, such as poverty status and race/ethnic group, is 
attached to performance records. Scores also can be aggregated to grade, school, and district 
levels. 
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Table 10 

Student Achievement Tests Used in the Four Study States by Test Name and Grades and Subject 
Areas Tested 





Arkansas 


Louisiana 


New Mexico 


Texas 


Tests 


• State benchmark 


• Louisiana Educational 


• Terra Nova (norm- 


• Texas Assessment 




tests (criterion- 
referenced) 

• Stanford 
Achievement Test, 
9th edition (SAT-9) 
(norm-referenced) 


Assessment Program for 
the 21st Century (LEAP 
21) (criterion-referenced) 

• Iowa Tests of Basic Skills 
(ITBS)/Iowa Tests of 
Educational Development 
(ITED) (norm- 
referenced) 


referenced) 


of Academic Skills 
(TAAS) (criterion- 
referenced) 

• Texas Assessment 
of Knowledge and 
Skills (TAKS) 
(criterion- 
referenced) 


Grades 


• 4, 6, 8 — State 
benchmark tests 
• 5 , 1 , 10 — Stanford 
Achievement Test, 
9th edition (SAT-9) 


• 4, 8 — Louisiana 
Educational Assessment 
Program for the 2 1 st 
Century (LEAP 21) 

• 3, 5, 6, 7, 9 — Iowa Tests 
of Basic Skills 
(ITBS)/Iowa Tests of 
Educational Development 
(ITED) 

• 10, 11 — Graduation Exit 
Exam 


• 3-9 — Terra Nova 


• 3-8, 10 — Texas 
Assessment of 
Academic Skills 
(TAAS) 

• 3-1 1 — Texas 
Assessment of 
Knowledge and 
Skills (TAKS) 


Subjects 


• Math, reading, 


• English language arts, 


• Math, reading, 


• Math, reading, 




writing — State 


mathematics, science, 


language arts, 


writing, science, 




benchmark tests 
• Complete 
battery — Stanford 
Achievement Test, 
9th edition (SAT-9) 


social studies — Louisiana 
Educational Assessment 
Program for the 2 1 st 
Century (LEAP 21) 

• Reading, language, 
mathematics, science, 
social studies — Iowa 
Tests of Basic Skills 
(ITBS)/Iowa Tests of 
Educational Development 
(ITED) 


science, social 
studies 


social studies 


Notes 


• For the State 


• Testing of different grade 


• The Terra Nova 


• The Texas 




Benchmark test, 


levels and subject areas 


changed formats in 


Assessment of 




4th grade was 


have been phased in 


2002-2003. 


Knowledge and 




added in 1999, 8th 


gradually since 


• The state plans to 


Skills replaced the 




grade was added 
in 2000, and 6 th 
grade was added 
in 2001. 


1999-2000. 


replace the Terra 
Nova with a 
criterion-referenced 
exam (grades 3-9, 
11) by 2005-2006. 


Texas Assessment 
of Academic 
Skills in 
2002-2003. 
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Data Quality. Due to increased attention to accountability needs, all four study states 
have been improving their capacity to measure student performance through standardized tests. 
While this process has improved the quantity of these data with regard to the number of tests 
offered and the grades tested, it also has resulted in inconsistency in the test scores available 
from year to year. In each of the four study states, changes were made during the study period to 
the tests administered, grades tested, or scoring standards. These changes are important for 
researchers to consider, especially in applying these data to analysis across time. Data are 
reported through state agency Web sites in all study states except New Mexico. These publicly 
available data, however, are limited for use in policy research because they are not disaggregated 
to reflect individual level scores or, in some states, subgroup populations. These data also are 
reported using aggregated scoring formats (e.g., percentage of students scoring at or above 
proficient level) of limited value to researchers. Researchers can request data from state 
departments of education that provide more refined measures of student achievement than those 
available on agency Web sites. 

Student, School, and District Characteristics. State education databases contain student, 
school, and district demographic data. Key demographic measures in the four states are listed in 
Table 11. Student poverty is measured by student participation in the free and reduced-price 
lunch program. All study states except New Mexico have collected these data in a consistent 
manner during the study period. Student race/ethnicity, gender, special education status, and 
participation in limited-English-proficient or gifted and talented programs are collected in 
education databases in all four states. School characteristics available in state databases include 
the type of school, the grades taught, student enrollment, school accountability ranking, and 
aggregated student characteristic data such as percentage of minority or high-poverty students. 
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District characteristics of interest to policy researchers include district wealth, geographic locale, 
district enrollment, and district/school accountability rating. District wealth is calculated as part 
of state funding formulas and is available for all four study states. Geographic locale (urban, 
rural, suburban, etc.) is available for Texas districts in state databases. A more standardized 
source of this information, however, is located at the federal level. The National Center for 
Education Statistics Common Core of Data produces a geographic locale label for each school 
and district in all states in the nation. 



Table 1 1 

Unit of Analysis Possible for Student, School, and District Characteristics in the Four Study 
States 



Measure 


Arkansas 


Louisiana 


New Mexico 


Texas 


Poverty status (free and reduced- 


Grade 


School 


Grade 


Grade 


price lunch program participation) 


School 


District 


School 


School 




District 




District 


District 


Race/ethnicity 


Grade 


School 


Grade 


Grade 




School 


District 


School 


School 




District 




District 


District 


Gender 


Grade 


School 


Grade 


Grade 




School 


District 


School 


School 




District 




District 


District 


Special program participation 


Grade 


School 


Grade 


Grade 


(special ed., bilingual. Title I, etc.) 


School 


District 


School 


School 




District 




District 


District 


School type (elementary, high, etc.) 


School 


School 


School 


School 


Grade range 


School 


School 


School 


School 


Total enrollment 


School 


School 


School 


School 




District 


District 


District 


District 


Per pupil expenditure 


District 


School (estimated) 


District 


School 






District 




District 


District wealth 


District 


District 


District 


District 


Geographic locale 


N/A 


N/A 


N/A 


District 


Accountability ranking 


N/A 


School 


School 


School 






District 




District 



Note. Free and reduced-price lunch data in New Mexico were reported inconsistently from 1999-2003. 
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Community Characteristics. Education databases do not collect information about 
characteristics of the community within which a school and/or district is located. Census 
information provides a number of relevant indicators for community well-being, including 
family income, parent education levels, and receipt of public assistance. 

Summary 

SEDL researchers found that existing state data can be used for education policy research 
on instructional resource allocation. Dollars spent to support instruction can be examined using 
fiscal data broken down by function, object, and program (in three of four study states) 
categories. These dollars also can be examined through staff databases that contain individual- 
level staff salaries. The added benefit of using individual staff salary data is the possibility of 
linking salaries to staff characteristics, such as years of experience, highest degree, and 
certification information. These staff characteristics, however, must be used with caution due to 
data quality concerns. Researchers should check staff years of experience data over multiple 
years to detect and correct errors. Also, since certification databases are managed separately 
from other staff characteristics, these data must be merged carefully with other staff data to avoid 
misaligned data. Certification data are also limited because data are continually updated in 
certification databases, and year-to-year snapshots of certified staff must be derived from 
certification issue dates (if available). Staff resources can be studied using state databases, and 
full-time equivalency (FTE) counts (head counts in Arkansas) on a wide range of staff categories 
are available in staff databases. These counts can be matched with staff demographic or other 
characteristics or calculated to create ratios such as pupikteacher or teacher:administrator. Actual 
class size information, however, is reported only in Louisiana. 
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Student performance data and information on student, school, and district characteristics 
are essential for conducting research on resources and student performance. Student performance 
data in each of the states are unique and have undergone changes in years tested and/or the tests 
administered, creating difficulty for researchers to attempt regional comparisons or longitudinal 
analysis. Student performance scores are useful measures because they can be aligned to student 
characteristics and grade- or school-level resource data. Additionally, these data reflect measures 
that are familiar to state policy audiences. Student, school, and district characteristics are 
available in education databases in all four study states. These data are of reliable quality with 
few exceptions in some states and are of critical value in understanding the relative importance 
of student, school, and district environments on resources and student performance. 

Policy questions of interest to the southwestern region and nationally can be answered 
using existing state education databases. By merging data on resources, student performance, and 
student, school, and district characteristics, researchers can answer policy questions such as the 
following: 

• What are effective allocation patterns of instructional dollars in schools and districts with 
varying characteristics? 

• Is teacher pay linked to teacher quality and student performance? 

• What are the recommended distribution patterns of staff with differing characteristics? 

• Should resources be invested to lower class size? 

Although existing state education data offer great potential for informing policy through 
research, these data also pose opportunities for state policymakers to improve them for future 
research. Fiscal data could be improved by adding a measure for individual staff benefits in three 
states (Arkansas already collects these data) so researchers could consider the total compensation 
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package paid to teachers and other staff. States should collect information on professional 
development for teachers. Data on professional development spending would greatly increase 
researchers’ ability to understand what investments in this area are effective. In order to better 
address priorities raised by the federal No Child Left Behind legislation, states could improve the 
scope and quality of teacher data. State policymakers and data managers should work to improve 
the reliability of staff experience data and the usability of teacher certification data. In order to 
better measure teacher quality, states should collect information about teacher degree major and 
professional development and should standardize data on teacher test scores. To understand the 
relationship between class size and student performance, researchers need more accurate 
estimates of actual class size that go beyond pupikteacher ratios. 

Implications for Future Research. Based on this study of existing state education data, we 
conclude that there is a need for increased attention on and use of these data for policy research 
purposes. Key features of the fiscal, staff, and student data we identified in this study helped 
frame SEDL’s next resource allocation study. Student performance data are available in the form 
of criterion-referenced and norm-referenced test scores in all four study states, which provide 
outcome measures for a resource allocation study. This study also identified teacher salary data 
as a reliable and flexible subset of state financial data that all states collect at the individual level. 
Teacher characteristics collected in state databases are critical to understanding salaries and 
student performance. Several key measures of teacher qualification are available in all four of the 
study states, including years of experience, highest degree, degree institution, and certification. 
Finally, data on the socioeconomic environments where teaching and learning occur are 
collected by state education agencies in the form of student demographics and school and district 
characteristics. Based on this understanding of data available in existing state databases, we 
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propose to focus our next study on the question “What is the relationship between teacher salary 
levels and student achievement in high-need schools?” This study will make use of existing data 
and provide critical information to state and local policy audiences regarding the allocation of 
teacher resources. 
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Chapter 4 

Recommendations — What Can Education Policymakers, State Data Managers, and 
Researchers Do to Increase the Use and Quality of Existing State Education Data? 

State education databases are an important source of information that researchers and 
policy audiences should not overlook. Although state agency data-request procedures and the 
usability of some variables present challenges, the potential utility of applying these data to 
policy research questions is very high. Based on study findings, we recommend that 
policymakers, state data managers, and researchers work together to expand their use of state 
education data for resource allocation research. We also recommend that policymakers and state 
departments of education support and implement data improvements for greater applicability to 
informing state policy and find ways to make these data more available and accessible for 
research use. 

Expand the Use of State Education Data 

Our assessment of data use in the four study states clearly showed that data have not been 
used to their fullest potential to support policy decisions. Specifically, data use has been 
extremely limited to a few areas of interest such as school funding formulas, adequacy of 
education funding, and descriptive studies of finances and teachers. Policy studies that help 
decision makers understand the relationship between resources and student performance are rare 
in the four study states. Given what we learned from this study, policymakers and researchers 
need to work together to become more familiar with these data and to use them to inform 
decisions. Increased use of existing state education data not only would result in increased 
information for policymakers but also would fuel a feedback mechanism for states to better 
understand how data need to be improved or expanded to serve information needs. 
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Improve State Education Data for Policy Research 

Applying state education data to policy research purposes is a relatively recent priority 
that is not fully recognized by states even today. In order for the data needs of policy research to 
move to the forefront, state policy audiences, data managers, and researchers must provide input 
on how existing data could be improved and changed for research purposes — in addition to more 
traditional reporting and monitoring purposes. Policy audiences need to expand their use of 
existing data to inform their decisions by learning about what data exist and requesting that these 
data be used to answer critical policy questions . 6 Data managers can work to improve state data 
collection systems and increase the accessibility of data. Researchers can support policy 
audiences by conducting more research with existing state data and providing feedback to data 
managers regarding the quality and accessibility of these data. Although expanding the use of 
existing data for research purposes is an important goal, policymakers, data managers, and 
researchers also must be aware that state education data are collected for competing needs such 
as federal reporting, tracking state accountability goals, and supporting state funding formulas. 
When considering improvements and changes to state databases, we must balance the time and 
resource burdens that changes in state data systems create for schools, districts, and state 
agencies. 

According to previous data assessment studies, policymakers, data managers, and 
researchers should consider several criteria to determine whether state data can be used to 
support their research and information needs regarding instructional resources, including the 
following: 



6 Although this report highlights the usability of existing state education databases, policy 
researchers should continue to seek alternative sources of data and/or sponsor targeted collection 
of data if state sources are not sufficient. 
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1 . Data must be available and accessible through a central source or through 
collaboration between agencies or departments that compile data related to instructional resource 
allocation. Data must be current and released to users in a timely manner. 

2. Ideally, data systems would be automated, interconnected, and contain appropriate 
controls for confidentiality protection (Cohen, 1997). Privacy and confidentiality policies should 
be clear and consistent. 

3. Data must be consistent, valid, and reliable, and reporting procedures should be 
enforced using such tools as standardized software programs. 

4. Data systems must be user-friendly and supported by technicians who are well- 
trained and knowledgeable about data, hardware, and software. Users must have access to 
detailed data documentation. 

5. Data systems must include data on all levels of the system (classroom, school, 
district, and state). Ideally, researchers should be able to disaggregate data to any level of 
organization. 

6. Data systems must measure a wide range of instructional resources and costs, 
accurately gauge state performance expectations, and be able to relate instructional resource data 
to student performance and demographic information (Busch & Odden, 1997; Farland, 1997; 
Pane & Zwillinger, 2002). 

It is clear from this study that education data across the four study states share similar 
structures and common variables. Federal reporting requirements linked to specific federal 
programs (Title I, Title II, special education programs, etc.) and national education data 
collection efforts (U.S. Census, National Center for Education Statistics, etc.) have helped to 
create similar data collection processes among states. As this study revealed, however, critical 
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differences in the variable definitions used by each state and the range of information they 
collect leave researchers with few avenues for pursuing cross-state or regional studies on 
education resources. Policymakers, data managers, and researchers should maintain a dialogue 
with national data centers such as the National Forum for Education Statistics that attempt to 
bridge the gap between the unique needs of state data systems and the research benefits of 
establishing national data standards. 

We also recommend that policymakers and data managers consider the following 
targeted improvements to increase the usability of education data for resource allocation 
research: 

1. Instructional expenditures at the school level are collected currently only in Texas; 
the other three study states collect this information at the district level. Adding school-level 
detail of how instructional resources are allocated would enable policy researchers to consider 
spending needs of schools in varying environments. This is especially important for those states 
with large or diverse school districts. If policy audiences are to understand how spending 
differences within districts affect the success of students in different schools, these data must be 
tracked and compiled in state education databases. However, when establishing new accounting 
mechanisms for schools and districts to use for reporting school-level expenditures, collection 
methods should be designed to avoid simple proration of district expenses across schools or 
inconsistency in coding expenditures across sites. 

2. Teacher quality is quickly becoming one of the highest policy priorities due to the 
federal No Child Left Behind legislation and research results emphasizing the importance of 
good teachers as a predictor of student success. Policymakers and data managers need to ensure 
that data collected on teacher qualifications align with federal priorities. In all four study states, 
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state teacher databases lack information about a teacher’s degree major. All four states also track 
teacher certification information on a cumulative basis, so snapshots in time of the existing 
teacher pool are difficult or impossible to obtain. We also recommend that data managers 
improve teacher certification data so that certifications can be aligned easily to teacher subject 
areas and grade levels. Teacher experience is a critical measure for policy decisions on teacher 
compensation, and data managers need to improve the accuracy of these data. 

3. Class size limits are imposed by both federal and state policy, and the benefits of 
smaller class sizes have been the topic of intensive study over recent years. In order for state 
policy audiences to fully understand the influence of class size on student performance in diverse 
schools and districts, reliable data must be available. State education data in each of the four 
study states enable users to create different measures for class size. In three of the four states, 
students are not linked to their classroom teachers, so a true estimate of class size cannot be 
created. In New Mexico data do link students to teachers, but access to these data is restricted 
from outside users. If policymakers are to fully understand the relationship between class size 
and student success, accurate measures, including data that link individual teachers to specific 
students or classrooms, must be created for use in policy analysis. 

4. Professional development is currently unaccounted for in state education databases in 
the four study states. Current research suggests that high-quality professional development for 
teachers affects the success of teachers (Charles A. Dana Center, 2002; Porter, Garet, Desimone, 
Yoon, & Birman, 2000). The No Child Left Behind legislation also has increased attention on the 
need for high-quality professional development for teachers, and states must account for the 
number of hours teachers spend in professional development. Additionally, without data on the 
amount and type of professional development for teachers and the cost of investments in 
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professional development to schools and districts, policymakers cannot consider the costs and 
benefits of statewide initiatives to provide professional development to educators. 

Improve Accessibility of Data for Outside Users 

A number of factors influence the accessibility of state education data for use in policy 
research. If data are to be shared with outside users such as policy audiences and researchers, 
accessibility and availability are the critical first steps. We recommend policymakers and state 
education agencies consider the following issues 

1. Interpretations of federal Family Educational Rights and Privacy Act (FERPA) 
regulations vary from state to state and change over time, affecting the accessibility of 
individual-level data. Individual-level data are necessary to conduct in-depth analysis of student 
subgroups and relationships between different types of students, teachers, and resources. 
Policymakers and data mangers should ensure that FERPA regulations are interpreted in a 
consistent manner and should find ways for education agencies to share data from state databases 
while ensuring confidentiality of individuals. 

2. Agencies that house state education data should ensure that procedures and staff are 
in place to assist data users. States vary in their ability to respond to requests from outside data 
users. Three of the four study states have limited staff who can pull data, address confidentiality 
concerns and scramble unique identifiers, and merge databases as needed to respond to data 
requests. In Texas, a standard process for responding to outside data requests has been instituted. 
However, in order to support the work that these requests generate, that state also charges a 
significant fee for special data requests, restricting accessibility to those who can afford these 
fees. State education agency staff are a critical resource to data users for data requests, 
information about data structures and variables, and coordination with the multiple departments 
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or related agencies that collect and manage education data. Clear procedures for data requests 
should be established, and state data managers should communicate the time and cost needed to 
provide data. 

3. Departments at state education agencies that collect data should work toward creating 
centralized data systems that combine the multiple education databases within each state. State 
education data related to instructional resources are divided among multiple databases and are 
managed by different departments within state education agencies or other related agencies. Data 
users must navigate different database structures and different levels of documentation and staff 
time and expertise. Fiscal data generally are collected and compiled separately from staff and 
student data. Teacher certification data are collected and managed by a different agency in Texas 
and by distinct departments within the department of education in the other three study states. 
Decentralized data systems create difficulties in answering policy questions that align fiscal and 
staff resources with student performance. Efforts in Louisiana to compile all education data into 
a central system provide a model for aligning disparate data sources. The Louisiana Educational 
Accountability Data System (LEADS) is an ongoing effort to implement an integrated data 
management system to support Louisiana’s education information needs. The Louisiana 
Department of Education is phasing in the Louisiana Educational Accountability Data System 
over time and already has integrated multiple data collection and dissemination systems (e.g., 
School Transcript System, School and District Accountability, the Data Warehouse, and 
Minimum Foundation Program Accountability). The costs and technical expertise required to 
complete this type of data integration are not yet available, nor are assessments of the relative 
benefits or utility to potential users. Additionally, the Texas Public Education Information 
Resource (TPEIR) database is being developed as a cross-agency data management system that 
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combines primary, secondary, and higher education information. This database is jointly 
managed by the Texas Education Agency, the Texas Higher Education Coordinating Board, and 
the State Board for Educator Certification, and integrates data collected through several different 
systems. We recommend that state education agencies and policymakers in all states investigate 
similar initiatives in order to expand access to state education data. 

4. Improvements could be made so that data documentation is consistently available and 
comprehensive for all state education data. If researchers are to use data accurately, they need 
detailed documentation on what data variables are collected and computed by state education 
agencies. Information on variable definition, type, ranges (if applicable), and year-to-year 
changes should be made available. Agencies that manage education data should post updated 
documentation to agency Web sites to increase accessibility to this information. 
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Appendixes 

Appendixes A-E are included as reference information for researchers and data users. 
Appendix A provides a description of the methods used to conduct this study, including a 
description of the data quality criteria used to assess state data systems (see Table A3). 
Appendices B-E describe in fine detail the data collected and housed by state education agencies 
in Arkansas, Louisiana, New Mexico, and Texas. Each state section describes relevant databases 
and discusses the data available to measure instructional expenditures; staff characteristics; 
student performance; and student, school, and district characteristics. For ease of use, each state 
section follows an identical structure and sequence of tables, as listed below: 

Table XI Summary of Existing State Databases 

Table X2 Measure of Instructional Expenditures From District Financial Data 

Table X3 Strengths and Challenges of Instructional Expenditure Data That Affect Their Use 
for Policy Research 

Table X4 Staff Characteristics Available in State Databases 
Table X5 Student and Staff Counts and Ratios 

Table X6 Strengths and Challenges of Staff Data That Affect Their Use for Policy Research 
Table X7 Student Performance Tests 

Table X8 Strengths and Challenges of Student Performance Data That Affect Their Use for 
Policy Research 

Table X9 Data Available in State Education Databases on Student Characteristics 
Table X10 Data Sources for School and District Characteristics 
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Appendix A 
Methodology 

SEDL researchers examined existing state data to assess their usability for conducting 
research on fiscal and staff instructional resources; student performance; and student, school, and 
district characteristics. The five states in SEDL’s region (Arkansas, Louisiana, New Mexico, 
Oklahoma, and Texas) were prospective sites for the study. SEDL contacted the state 
departments of education in all five states to invite them to participate. Four of the five states 
agreed to participate; Oklahoma declined. In New Mexico, organizational changes at the New 
Mexico Public Education Department prevented a timely transfer of sample data for analysis. 

Due to the variability between state data, we conducted data analyses separately within each 
state. We also considered the state-specific policy concerns regarding the instructional resource 
allocation questions of interest to this study by consulting with state education agency officials in 
each state and assessing how state data have been used to inform policy in these states. 

Purpose of the Study and Research Question 

For this study, we addressed the question “Do state databases allow the investigation of 
the relationship between fiscal and staff instructional resources and student performance?” We 
examined state data in participating study states to discern whether these data supported new, 
rigorous research about resources and student performance. In order to assess the capacity of 
existing state data to conduct such research, we (a) identified the key variables within fiscal and 
staff instructional resources; student performance; and student, school, and district 
characteristics; (b) developed criteria to assess the usability of these data; and (c) applied these 
criteria to identify resource allocation questions that can be answered with state data. We also 
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examined whether commonalities exist in these data across the study states. Five major research 
activities guided this work, as described below and summarized in Table Al. 

Initial Data Identification and Examination 

We identified and examined data from state departments of education that measure 
instructional expenditures; staff characteristics; student performance; and student, school, and 
district characteristics. We focused attention on data available during a 4-year study period 
(1999-2003) and used public information sources such as online data descriptions and reports to 
determine what databases and types of measures were available in each study state. We 
communicated with state data managers through telephone, e-mail, and face-to-face visits to gain 
detailed descriptions of databases and definitions of relevant variables. Table A2 provides a list 
of fiscal, staff, and other measures we used to narrow the range of variables to investigate. We 
found that each state’s data varied with regard to the variables available, units of measure, and 
level of aggregation. 

Policy Research Needs on Resource Allocation 

We conducted an assessment of state data utilization in the four study states to determine 
the extent to which these data have been used to support resource allocation decision making 
(see chapter 2 of this report). We identified reports and research studies that used state data on 
education resources through Web searches, discussions with state policymakers, and searches of 
relevant databases, such as the Education Resources Information Center (ERIC). We also 
engaged state policy audiences in discussions at SEDL’s 2003 policy forum and in face-to-face 
discussions during data collection visits to identify specific policy concerns. Based on these 
activities and findings from our initial identification of relevant resource allocation variables in 
state databases, we identified a subset of policy questions to consider for this investigation. 
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Table A1 

Summary of Research Activities 



Activity 



Description 



Objective 



1 . Identify relevant Researchers reviewed public 

existing information sources about existing 

databases and databases and relevant variables, 

data variables of 

interest 

Researchers conducted phone 
interviews and face-to-face visits 
with representatives of 

• state departments of education 

• related state agencies (e.g., teacher 
certification) 



2. Identify policy Researchers assessed how state data 
research have been used to support resource 

questions about allocation decisions, 

resource 

allocation 

Researchers facilitated discussion at 
SEDL’s policy forum with 
representatives from 

• state departments of education 

• related state agencies (e.g., teacher 
certification) 

• state legislature 

• governor’s office 



3. Assemble data 

and determine 
usability of data 
for policy 
research 

4. Determine data 
capacity for 
supporting new 
rigorous 
instructional 
resource 
allocation 
research 



Researchers applied established 
criteria to assembled data and 
analyzed data using descriptive 
statistics. 



Researchers considered the 
usability of data and research needs. 



• Identify existing databases of interest for 
this study. 

• Confirm that fiscal, staffing, performance, 

and demographic data are available in state 
databases. 

Collect initial information on relevant 

variables, including 

• description of variables (level of analysis, 
longitudinal data, etc.) 

• definitions of variables (changes in 
longitudinal data, calculated variables, etc.) 

• information about supplemental sources of 
data 

• measurement of study variable not available 
in state data sets (e.g., estimating class size) 

• Identify policy issues that have already been 
addressed with the support of state data in 
each of the study states. 

• Understand the need for additional policy 

analysis using state education data. 

• Identify policy concerns related to 
instructional resource allocation in each 
study state. 

• Establish whether instructional resource, 
student performance, and demographic data 
have already been analyzed by researchers 
or policy audiences. 

• Discuss upcoming policy challenges and 
ways that state data could inform them. 

• Assemble data variables relevant to the 
resource allocation policy questions. 

• Determine initial usability of data. 

• Assess data for appropriateness for more 

sophisticated data analysis. 

• Assess to what extent existing state data can 
be used to answer instructional resource 
allocation questions and what questions 
could be answered if data were improved. 
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5. Disseminate 
findings to 
policy audiences 

decision making. 

• Raise awareness of how data can contribute 
to instructional resource allocation 
decisions. 

• Discuss possible uses of information, 
including improving data management 
systems, prioritizing policy issues/areas 
related to instructional resources, and 
conducting new research on instructional 
resource allocation. 



Researchers are disseminating 
findings in written reports and 
discussions at meetings convened 
with policy audiences in each of the 
study states. 



Present findings and assessment of the type 
and scope of research that can be conducted 
to answer the instructional resource 
allocation questions. 

Get feedback on implications for state policy 
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Table A2 

Study Variables and Data Sources 



Major Variable Category 


Variable Descriptions 


Data Sources 




Operation expenditures for activities 


State departments of 




dealing with the interaction between 


education data 




teachers and students: 


disaggregated to the 




• function, program, and object 


lowest level of detail 


Fiscal 


expenditures related to instruction, 


possible 


expenditures 


instructional staff support, and student 
support 

• payments from all funds for salaries, 
employee benefits, supplies, 
materials, and contractual services 






Data variables related to teachers, 


State departments of 




aides, and administrators that include 


education and other 


Instructional 


information: 


supplemental state data 


resources 


• reflected as head counts, full-time 


disaggregated to the 


Staff 


equivalency counts, and ratios (e.g.. 


lowest level of detail 


pupihteacher, teacher:administrator) 


possible 


characteristics 


• about classroom assignment (e.g., 
subject area and grade level) 

• on teacher characteristics (e.g., 
teacher experience, certification, and 
education type/degree level) 




• Data from state achievement tests 


State departments of 


Student performance 


(norm- and criterion-referenced tests 
administered by state) 


education 




Exogenous characteristics that describe 


State departments of 




the students, schools, and districts: 

• student socioeconomic status 

• student race/ethnicity 

• school size 


education 


Student, school, and 


• district size 




district characteristics 


• district wealth 

• locale 

• family income 

• parental education 

• social services available 
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Determine Usability of Data for Policy Research 

Once we identified data and refined and prioritized a list of variables for each state, we 
examined and determined their usability on a range of criteria (Table A3). The following three 
activities informed the data usability assessment: 

1. We discussed databases with state data managers in order to gauge availability, 
accessibility, completeness, accuracy, consistency, and alignment. 

2. We examined data elements, focusing on completeness, accuracy, consistency, and 
alignment. 

3. We identified those data elements that were usable for further analysis and used 
descriptive statistics (frequencies, means, ranges, etc.) to gauge variability of data elements, 
show the level of disaggregation possible, determine if units of measure can be modified to fit 
specific analysis needs (e.g., staff full-time equivalency [FTE] counts vs. head counts), and 
discern whether missing data can be calculated or imputed. We also identified related variables 
that would align fiscal, staffing, performance, and demographic data. 

We considered the results from our examination of education data in the study states to 
determine how they can be used to conduct new rigorous research about instructional resources, 
student performance, and demographic characteristics. For example, if instructional resources 
cannot be aligned with student performance data, researchers are limited in methods of 
answering the question of whether instructional inputs influence student achievement. We 
applied our knowledge about state data (e.g., variability of data points, levels of disaggregation, 
and availability of longitudinal data) to determine whether informative statistical approaches 
could be applied to these data (cohort analysis, longitudinal study, etc.). We also gained insights 
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about how state data can be improved to better address the goal of supporting instructional 
resource allocation research. 



Table A3 

Data Usability Criteria 



Criterion 


Indicators 


Information Source 


Availability and 
accessibility 


Do sufficient data exist to measure instructional resources, 
student performance, and demographic characteristics at 
a useful level of analysis? 


• Codebooks and/or visual 
examination of data 
points 




Do data reflect current (2002-2003) measures? 

Are data contained in accessible formats? 

Do state policies regarding confidentiality balance privacy 
concerns and need for access? 


• Discussion with state 
data managers 


Completeness 


Are data complete? 

Do data measure all levels of the education system 
(classroom, school, district, and state)? 


• Codebooks and/or visual 
examination of data 
points 

• Examination using 
descriptive statistics 
(frequency, ranges) 




Has completeness been maintained through standardized 
data collection procedures? 


• Discussion with state 
data managers 


Accuracy 


Do data accurately measure intended variables? 


• Codebooks and/or visual 
examination of data 
points 

• Examination using 
descriptive statistics 
(frequency, ranges) 




Has accuracy been maintained through data cleaning, 
editing, calculations, and storage? 


• Discussion with state 
data managers 


Consistency 


Is consistency evident among student-, school-, and 
district-level data? 

Do data reflect uniform use of definitions? 

Are data consistent over time so that comparisons can be 
made longitudinally? 


• Codebooks and/or visual 
examination of data 
points 

• Examination using 
descriptive statistics 
(frequency, ranges) 




Has consistency been maintained through uniform 
reporting procedures? 


• Discussion with state 
data managers 


Alignment 


Are there common data elements that can link fiscal, 
staffing, performance, and demographic data systems? 

Do data identify each record unit by all standardized 
identifiers that apply to the unit (e.g., do school-level 
data elements contain the school name, state ID for the 
school, and district ID for the school)? 


• Codebooks and/or visual 
examination of data 
points 

• Examination using 
descriptive statistics 
(frequency, ranges) 
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Dissemination of Study Findings 

These findings are of importance to state and local education decision makers, education 
researchers, and data managers in each of the study states. We will disseminate findings through 
written reports, summary policy papers, and meetings with targeted audiences in the four study 
states. 

Limitations of this Study 

A number of limitations impacted this study. Resource, performance, and demographic 
data are collected by school districts and managed by state data managers. We could not fully 
account for the validity and reliability of these data from diverse district-level recording methods 
and other sources of data such as teacher certification applications and test scoring entities. 

States continually make changes and additions to their data systems; consequently, a point-in- 
time assessment of data capacity is useful for only a short time, after which updates are required. 
Finally, the time and capacity of data managers and other staff to assist data users varied widely 
from state to state. State education agency procedures for responding to data requests varied 
widely, ranging from formal procedures to informal practices to lack of capacity to provide data. 
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Appendix B 

Arkansas State Education Data 

The Arkansas Department of Education is the primary source of education data for the 
state. The department oversees the provision of education services in over 1,100 schools 
organized into 310 districts. 7 Information is collected on financial activity in each school district, 
on the characteristics of more than 33,000 teachers, and on the performance and demographics of 
over 400,000 students per year. Data collection and management have undergone restructuring in 
the recent past with improvements made to data accessibility, collection, and reporting. 

Overview of Existing State Data 

Three major databases are relevant for policy research studies focusing on instructional 
resources and student performance. Financial, staff, and student data are collected through the 
Arkansas Statewide Information System (SIS); staff certification information is compiled in the 
Arkansas Professional Licensure System (APLS); and student performance data are managed by 
the Arkansas Comprehensive Testing, Assessment, and Accountability Program (ACTAAP) (see 
Table AR1). 

The Arkansas Statewide Information System (SIS) was implemented by the Arkansas 
Department of Education to manage data collection and reporting activities for school districts in 
the state. The Arkansas Statewide Information System is the data collection vehicle for financial, 
staff, and student information, and it also provides a computerized interface for schools and 
districts to access their data. This system resulted from legislation requiring a reduction in the 
amount of paper used for data reporting and the need for accurate and timely information for 

7 Recent consolidation reduced the number of districts to 308 in 2003-2004 and to 254 in 
2004-2005. 
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policymakers and school systems. Currently, the Arkansas Statewide Information System is 
managed by the Arkansas Public School Computer Network (APSCN) at the Arkansas 
Department of Education. Of interest to this study are the following Arkansas Statewide 
Information System data elements related to district finances and staff and student information: 

1. Financial data are collected and reported by districts through the Arkansas Statewide 
Information System. Districts enter data into the system using variable categories aligned with 
the U.S. Census reporting system (F-33 form), including revenues by source and expenditures by 
function and object as listed in the Financial Analysis and Accountability Report (FAAR) 
template, which is located on the Arkansas Public School Computer Network ( APSCN) Web site 
(http://www.kl2.ar.us/apscndocs/finrptcard.pdf). Financial Analysis and Accountability Reports 
are generated for each district separately and can be downloaded as portable document format 
(.pdf) files at http://www.as-is.org/search/list_reports.html. The Public School Finance and 
Administration Unit at the Arkansas Department of Education provides technical assistance to 
school districts regarding financial reporting procedures and standards and maintains the 
Arkansas School and Educational Service Cooperative Financial Accounting Manual. 

2. The Arkansas Statewide Information System is also the statewide student and staff 
information system for K-12 public education in Arkansas. School districts are asked to collect 
and submit detailed student and staff information for inclusion in the Arkansas Statewide 
Information System database. Data are collected on all public school students and on all certified 
and classified staff. Districts are required to submit student data on six of seven reporting periods 
starting on September 15 (Cycle 1) and ending on June 7 (Cycle 7). Staff data are reported on the 
second reporting period and financial data are reported on the first reporting period. School-level 
staff and student information can be accessed through a file transfer protocol site, 
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ftp://165.29.215.34/. All data are in Microsoft Access format. Arkansas Statewide Information 
System database variables are listed and defined in two handbooks, Statewide Information 
System and Reporting and Statistical Database Schema, which are updated annually by the 
Arkansas Department of Education and available by request. 

The Professional Certification/Licensure Unit at the Arkansas Department of Education 
maintains the Arkansas Professional Licensure System (APLS), a database that contains 
certification information about each individual teacher and administrator in the state. Information 
in this database is collected from applications for licensure, teacher test results, and certifications 
granted to applicants. Documentation of the Arkansas Professional Licensure System data is 
limited and is available by request from the Professional Certification/Licensure Unit. Data on 
teacher and administrator licensure can be requested from the Professional 
Certification/Licensure Unit. A limited amount of these data are transferred to the Arkansas 
Statewide Information System and may be accessed in aggregated reports through a file transfer 
protocol site (ftp://165.29.215.34/Certification/) maintained by the Information and Technology 
Unit at the Arkansas Department of Education. 

The Arkansas Department of Education Accountability Unit under the Arkansas 
Comprehensive Testing, Assessment, and Accountability Program oversees the state’s student 
achievement testing (http://arkedu.state.ar.us/actaap/index.htm). The state administers multiple 
student achievement tests, including the Stanford Achievement Test 8 norm-referenced exam, a 
criterion-referenced state benchmark exam, and end-of-course exams. Data are collected and 
compiled on all of these tests for all students that take the exam(s). School, district, and state 
performance data are publicly accessible through electronic report cards on the Arkansas School 

8 The state norm-referenced test changed to the Iowa Test of Basic Skills in 2004. 
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Information Site (AS-IS), http://www.as-is.org/. The National Office of Rural Measurement and 
Evaluation Systems at the University of Arkansas, Fayetteville supports the management of 
student achievement test data through contract with the Arkansas Department of Education. This 
university department also provides reports and summaries of student performance data on their 
Web site (http://normes.uark.edu/). 

Table AR1 

Summary of Existing State Databases, Arkansas 



Data Category 


State Database 


Managing Agency/Department 


Instructional 


Arkansas Statewide 


Public School Finance and Administration Unit, 


expenditures 


Information System (SIS) 


Arkansas Department of Education 


Staff characteristics 


Arkansas Statewide 
Information System (SIS) 


Arkansas Public School Computer Network 
(APSCN), Arkansas Department of Education 


Student characteristics 


Arkansas Statewide 
Information System (SIS) 


Arkansas Public School Computer Network 
( APSCN), Arkansas Department of Education 


Student performance 


Student achievement data 


Accountability Unit, Arkansas Department of 
Education 


Teacher licensure 


Arkansas Professional 
Licensure System (APLS) 


Professional Certification/Licensure Unit, 
Arkansas Department of Education 



Instructional Spending 

Dollars spent for instructional purposes by school districts in Arkansas can be measured 
through two data sources. First, researchers can obtain financial data for each school district on 
spending in specific instructional areas. Second, researchers can request salary information on 
individual instructional staff members. These data sources are summarized in Table AR2 and 
described below. 
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Table AR2 

Measures of Instructional Expenditures From District Financial Data, Arkansas 



Fiscal Measure 


Variables Available 


Description 


Unit of 
Analysis 


Instruction-related 

expenditure 

functions 


Instruction 


Activities dealing directly with the 
interaction between teachers and 
students. Includes a breakdown of 
spending by program activity. 


District 




Support 

services — students 


Activities designed to assess and 
improve the well-being of students and 
to supplement the teaching process. 
Includes a breakdown of spending by 
service. 


District 




Support 

services — instructional 
staff 


Activities associated with assisting the 
instructional staff with the learning 
process. 


District 




Support 

services — general 
administration 


Activities concerned with establishing 
and administering district policy. 
Includes a breakdown of spending by 
administrative entity. 


District 




Support 

services — school 
administration 


Activities concerned with the 
administrative functions of a school. 
Includes spending for the office of the 
principal and other services. 


District 


Expenditure 


Personal 


Compensation paid to permanent and 


District 


objects 


services — salaries 


temporary district employees, 
including substitutes. 






Personal 

services — employee 
benefits 


Costs paid on behalf of employees that 
are not part of an employee’s gross 
salary. 


District 




Purchased professional 
and technical services 


Services performed by persons with 
specific expertise in a specialized field. 


District 




Purchased property 
services 


Services purchased to operate, repair, 
maintain, and rent property owned or 
used by the district. 


District 




Other purchased 
services 


Contracted services not classified in 
other object categories. 


District 




Supplies and materials 


Expenditures for supplies and 
materials. 


District 




Property 


Expenditures for the acquisition of 
land, buildings, improvements, initial 
equipment, and replacement 
equipment. 


District 




Other objects 


Amounts paid for good and services 
not otherwise classified. 


District 



Southwest Educational Development Laboratory 



73 




Investigation of Education Databases in Four States to Support Policy Research on Resource Allocation 



Staff salary data 3 Salary The contract amount paid to the Individual 

employee. certified/ 

classified staff 

Source of salary Fund code from which salary is paid Individual 

• Teacher salary certified/ 

• Operating classified staff 

• Building 

• Federal 

• Activity 

• Food Service 

Benefits The cost of employee benefits. Individual 

certified/ 

classified staff 

a Beginning in 2002-2003, salary information was linked to function and object financial categories and 
partial salary amounts were reported for specific program areas under which a teacher worked. 

Instructional Expenditures. The financial reporting system in Arkansas uses two function 
categories pertaining to instructional expenditures: instruction and support services. The 
instruction function encompasses spending on activities dealing directly with teaching students 
and teacher-pupil interaction. This function is divided into six subfunctions composed of 
specific program areas: regular programs, special education programs, workforce education 
programs, adult/continuing education programs, compensatory education programs, and other 
instructional programs (such as music, English as a second language, and gifted and talented 
programs). 

The support services function includes expenditures for services that provide 
administrative, technical, and logistical support to facilitate and enhance instruction and 
community services. The support services function is divided into four important subfunctions: 
(a) student, (b) instructional staff, (c) general administration, and (d) school administration. 
Neither the support services function nor the four subfunctions are divided into program 
categories as is done with the instruction function. The four subfunctions are defined as follows: 

1 . Student support services are defined as activities designed to assess and improve the 
welfare of students and supplement the teaching process. The student support services 
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subfunction is further divided into activity categories: student attendance, social work services, 
guidance services, counseling, health services, psychological services, speech pathology and 
audiology services, and physical and occupational therapy. 

2. Instructional staff support services include activities associated with assisting the 
instructional staff with the learning process. This sub function is further divided into curriculum 
supervision, educational media services, and other support services. 

3. General administration support services, although not directly related to instruction, 
provide important data for comparative purposes for policy analysis. This subfunction is divided 
into expenditures for different administrative entities, including district board of education, 
office of the superintendent, and other administrative offices. 

4. The school administration support services sub function includes spending on 
activities concerned with the administrative functions of the school, including the office of the 
principal. 

Object-level expenditures further describe the service or commodity obtained as a result 
of a specific expenditure. Eight object-level categories containing multiple subobject categories 
are used for financial reporting (see Table AR2). For example, the object-level category 
“personal services — salaries” is used for compensation paid to permanent and temporary 
employees of the district and also includes dollars spent for overtime, sabbatical, workshops, 
substitutes, and other expenses. “Personal services — employee benefits” includes costs paid on 
behalf of employees that are not included in the salary object. Expenditures on the district’s share 
of group insurance costs, teacher retirement contributions, tuition reimbursement, and other 
standard benefit categories (Medicare, Social Security, workers’ compensation, etc.) are included 
in this category. The remaining object-level expenditure categories are listed in Table AR2. 
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Individual Staff Salary Data. Aggregated expenditures on employee salaries and benefits 
are available using object-level categories in the financial data as described above. In addition, 
individual-level salary and benefits information for certified and classified staff are collected 
through the Arkansas Statewide Information System. Each staff record contains a total salary 
amount, an amount spent on benefits, and the source of revenue that funded the salary. As of 
2002-2003, these data were also linked to function and object financial categories. Also 
beginning 2002-2003 data were organized so that partial salary information could be obtained 
based on the specific program area(s) in which the teacher provided services. 

Data Usability. Researchers considered data on instructional spending from both the 
financial database and the Statewide Information System for their potential usefulness for policy 
research purposes (see Table AR3). Overall, the district-level data on instructional expenditures 
and the individual-level staff salary data are useful for conducting resource allocation research. 
The two sources of expenditure data provide both a district-level perspective of overall spending 
in instructional areas and individual-level information about salaries and benefits that could be 
aggregated for specific types of staff (e.g., teachers) and to the school or district. District 
expenditure data are easily accessed through the Arkansas Department of Education Web site, 
which provides reports in portable document format (.pdf) about each school district’s spending 
for instructional function and objects. Documentation is contained in the Arkansas Financial 
Accounting Manual, available by request from the Arkansas Department of Education, and in the 
Arkansas Statewide Information System handbooks that are updated annually. District financial 
expenditure data have been collected using consistent variable categories over the study period, 
and function and object categories match those reported to the U.S. Census. This consistency 
supports the completeness of these data as well as accuracy because districts are familiar with 
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state reporting formats. Staff salary information is collected for all staff, whether certified or 
classified. The Arkansas Statewide Information System was implemented with the goal of 
further standardizing and increasing the accuracy of the collection of staff data. The structure of 
the Arkansas Statewide Information System also allows easy alignment of staff salary 
information with staff characteristics, job assignment, and location. The alignment of these data 
allows users to average salary expenditures across staff roles, programs, schools, and districts. 

Data that measure instructional expenditures in Arkansas are not without challenges. 
Access to individual-level data from the Statewide Information System is limited to requests 
from the Arkansas Department of Education, whose staff have limited time and few standard 
procedures for providing data to outside users. Individual staff data are organized using staff 
Social Security numbers as a unique identifier, making confidentiality of these data an obstacle 
to data requests. Also hampering the availability of these data are the multiple departments 
within the Arkansas Department of Education that oversee data collection and compilation. 
Arkansas Statewide Information System data are primarily managed by the Information and 
Technology Unit; however, portions are maintained by the Arkansas Public School Computer 
Network, which previously was a different entity from the Arkansas Department of Education 
and is now under the purview of that department. Oversight for financial data collected by the 
Arkansas Statewide Information System is conducted by the Public School Finance and 
Administration Unit at the Arkansas Department of Education. Potential data users must 
understand the complexity of these relationships if data are to be accessed effectively. For 
example, data on individual staff benefits are collected from districts but are not reported in 
Arkansas Statewide Information System documentation. Therefore, in the case of this study, 
discussions with staff at the Arkansas Public School Computer Network were necessary to 
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understand these data. One important change in the collection of staff salary data had 
implications for the accuracy and consistency of these data. Beginning in 2002-2003, the state 
began collecting additional staff information on the portion of a staffs salary paid for work in 
specific program areas (partial salary variable). The state also redefined the total salary variable 
to measure the actual rather than contracted salary amount. Individual salary information was 
also linked to function and object expenditure codes from the financial data system. Although 
these changes created greater flexibility of use for these data and more refined and valid 
measures of individual salaries, these data were also problematic, as districts seem to be 
adjusting to new collection formats. 
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Table AR3 

Strengths and Challenges of Instructional Expenditure Data That Affect Their Use for Policy 
Research, Arkansas 



Criterion Strengths Challenges 



Availability 

and 

accessibility 


• Data are available from two sources: 
district-level financial data and 
individual-level spending on staff salaries 
and benefits. 

• Financial data reports are posted on the 
Arkansas Department of Education Web 
site and can be requested from that 
agency. 

• Data documentation is available by 
request; some is posted to the Arkansas 
Department of Education Web site. 


• Individual staff salary data are only 
available by request and must be 
stripped of identification number (SSN). 

• Oversight for different databases is 
divided among different units within the 
Arkansas Department of Education. 

• Not all available fiscal data are reported 
in Arkansas Statewide Information 
System documentation. 


Completeness 


• District financial data are available for all 
districts and all years of the study period. 

• Staff salary data are collected for all 
certified and classified staff. 




Accuracy 


• The consistency of the financial data 
structure supports greater accuracy of 
data collection from school districts. 

• The development of the Arkansas 
Statewide Information System has 
increased attention on accuracy and 
standard procedures. 


• Changes in reporting for individual staff 
salaries have not been fully integrated 
by district reporting. 


Consistency 


• Financial function and object categories 
are aligned to federal reporting categories 
and have not changed during the study 
period. 


• Individual-level staff salary data 
changed in 2002-2003 so that partial 
salaries based on program codes could 
be determined. These data also were 
linked to financial function and object 
expenditure categories, making the total 
salary an actual rather than contracted 
amount. 


Alignment 


• Individual staff salary data can be aligned 
with other staff characteristics. 

• Staff salary data can be averaged across 
staff roles, schools, and districts. 


• Financial data and individual staff salary 
data have been recently aligned with 
function and object codes, and the 
accuracy of this alignment is not certain. 



Staff Characteristics 

Information about staff characteristics is collected at the individual level in the Arkansas 
Statewide Information System. These data are collected at the October 1 5 reporting period each 
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school year. Demographic characteristics, qualifications, job assignment, and salaries (see 
previous discussion) are collected for all certified and classified staff positions. Additionally, the 
Professional Certification/Licensure Unit at the Arkansas Department of Education collects 
individual-level certification information on teachers and administrators (see Table AR4). 



Table AR4 

Staff Characteristics Available in State Databases, Arkansas 



Staff 

Characteristic 


Variables Available 


Unit of 
Analysis 


Source 


Job code 


Each job assignment performed 
by a staff person (multiple 
assignments per individual 
possible) 


Individual 


Arkansas Statewide Information 
System 


Location 


School and district to which the 
staff person is assigned 


Individual 


Arkansas Statewide Information 
System 


Experience 


Total years of experience in and 


Individual 


Arkansas Statewide Information 


(classified staff) 


out of state 




System 




Years experience in current 
district 


Individual 


Arkansas Statewide Information 
System 




Date hired by district 


Individual 


Arkansas Statewide Information 
System 


Education 

(teachers) 


Lowest degree earned 


Individual 


Arkansas Professional Licensure 
System 




Lowest degree institution 


Individual 


Arkansas Professional Licensure 
System 




Highest degree earned 


Individual 


Arkansas Professional Licensure 
System 




Highest degree institution 


Individual 


Arkansas Professional Licensure 
System 




Professional development hours 


Individual 


Arkansas Statewide Information 
System 


Demographics 


Gender 


Individual 


Arkansas Statewide Information 
System 




Race/ethnicity 


Individual 


Arkansas Statewide Information 
System 




Birth date 


Individual 


Arkansas Statewide Information 
System 
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Certification 


Subject area of license 


Individual 


Arkansas Professional Licensure 


(teachers) 


Grade level (lowest and highest 


Individual 


System 

Arkansas Professional Licensure 




grade) of license 




System 




Application type 


Individual 


Arkansas Professional Licensure 
System 




Licensure effective date 


Individual 


Arkansas Professional Licensure 
System 




Licensure expiration date 


Individual 


Arkansas Professional Licensure 
System 




Non-traditional licensure 


Individual 


Arkansas Professional Licensure 
System 




National board certification 


Individual 


Arkansas Professional Licensure 
System 


Grade 


Lowest grade level taught for 


Individual 


Arkansas Statewide Information 




each job assigned 




System 




Highest grade level taught for 


Individual 


Arkansas Statewide Information 




each job assigned 




System 



Counts and Ratios. Staff counts and ratios can be calculated from individual data in the 
Arkansas Statewide Information System database (see Table AR5). Head counts can be created 
for staff and grouped by job codes, schools, districts, qualifications, or demographic 
characteristics. Full-time equivalency (FTE) counts are not available in Arkansas staff data; 
although a variable exists for this information in the staff database, these data are not complete. 
Researchers can calculate a pupikteacher ratio using student enrollment information and an 
aggregate of the classroom teachers listed in the Arkansas Statewide Information System. State 
databases do not contain actual class size information. Staff ratios such as teacher: administrator 
or pupil: administrator can also be calculated using aggregated head counts from the Arkansas 
Statewide Information System data. 
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Table AR5 

Student and Staff Counts and Ratios, Arkansas 



Count/Ratio Measure 


Data Available 


Unit of Analysis 


Source 


Head counts of staff in specific 
roles (teacher, principal, etc.) and 
with specific characteristics (see 
Table AR4) 


Can be calculated 
from existing data 


Job code 

School 

District 

Demographic subgroups 


Arkansas 

Statewide 

Information 

System 


Class size 


N/A 


N/A 


N/A 


Other ratios: 

• pupihteacher 

• pupil: administrator 

• teacher: administrator 


Can be calculated 
from existing data 


School 

District 


Arkansas 

Statewide 

Information 

System 



Data Usability. As summarized in Table AR6, data on staff characteristics are useful in 
many ways for policy research purposes but also contain some limitations for these purposes. 
These data are collected on an individual level, creating flexibility for researchers to create 
aggregated data on a number of categories, such as school, district, and demographic 
characteristics, and to create student and staff ratios for schools and districts. Information on 
those staff with the job code of classroom teacher, for example, can be isolated for research 
purposes. The most important data usability challenge concerns accessibility of staff data for use 
by outside researchers. As presented in the discussion of staff salary data, state, district, and 
school summaries of staff information are available on the Arkansas Department of Education 
Web site, but individual-level data must be requested directly from the Arkansas Department of 
Education. Arkansas Department of Education data managers have limited time to respond to 
these requests, and few established procedures for providing raw data for outside users exist. 
Also, since the unique identifier for staff in the Arkansas Statewide Information System database 
is the individual’s Social Security number, data must be merged with other data if necessary and 
then stripped of identifiers before they can be sent to outside users. Another concern regarding 
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data usability is that teacher certification data are compiled from a different source and using 
different procedures than the Arkansas Statewide Information System. Data on the same 
individuals originate from two streams of data collection using different procedures, creating 
greater possibility of misaligned data on teachers. Additionally, licensure data are collected using 
a cumulative process in which updates are made to teacher information without annual archives. 
Researchers must rely on the certification issue date and expiration date of each staff in order to 
create a subset of active, certified staff for a given study period. Data managers at the Arkansas 
Department of Education explained that some teacher certification variables are unreliable. Years 
of experience data suffer from unreliable data collection practices, and data on national board 
certification are highly suspect for errors. Finally, although head counts of staff can be 
calculated, full-time equivalency (FTE) counts of specific staff categories cannot be calculated 
with existing data. A variable for full-time equivalency (FTE) count for certified staff does exist; 
however, the data are incomplete and cannot be used to estimate staff counts. 
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Table AR6 

Strengths and Challenges of Staff Data That Affect Their Use for Policy Research, Arkansas 



Criterion 


Strengths 


Challenges 


Availability 

and 

accessibility 


• Individual-level staff characteristics 
are collected with ability to aggregate 
data up to school or district levels. 

• Data documentation for the Arkansas 
Statewide Information System is 
updated annually. 


• Individual-level staff data are not 
publicly available and must be 
obtained by special request from the 
Arkansas Department of Education 
staff, who must strip or scramble the 
unique identifier (Social Security 
number). 


Completeness 


• Staff data are collected for certified 
and classified staff for all schools and 
all years of the study period. 


• Full-time equivalency (FTE) counts 
of staff categories cannot be 
determined due to incomplete data. 


Accuracy 




• Accuracy of certain variables of 
interest is suspect (e.g., years of 
experience and national board 
certification). 


Consistency 


• Data collection documentation 
provides clear instructions with 
helpful examples for the user. 




Alignment 


• Staff characteristics can be linked to 
salary data and licensure data. 


• Teacher data are a combination of 
two data collection efforts: staff 
characteristics and teacher 
certification. 

• The cumulative process used to keep 
the licensure database updated 
creates the potential for misaligned 
data. 



Student Performance 

The state of Arkansas uses multiple tests for measuring student performance: norm- 
referenced tests, criterion-referenced tests, and end-of-course exams (see Table AR7). Student 
performance is gauged at the school and district level and is the basis for the school report cards. 

Norm-Referenced Tests. Arkansas has traditionally relied on norm-referenced student 
achievement tests to track performance of students. The Stanford Achievement Test (SAT), 9th 
edition, has been in place since 1996 for grades 5, 7, and 10. The state reports percentiles for the 
complete battery and disaggregated data for math, reading, language arts, and other subject areas. 
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Criterion-Referenced Tests. Based on 1999 state legislation, the criterion-referenced state 
benchmark tests were introduced gradually for grades 4, 6, and 8 in reading, writing, and math. 
Since 2001-2002 all three grade levels have been tested. Data are reported in scale and raw 
scores. 

End-of-Course Exams. End-of-course exams in algebra, geometry, and literacy are 
administered after course completion. There is no high school exit exam currently in place in 
Arkansas. 

State Accountability Ranking. Arkansas currently does not rank all of its schools and 
districts for accountability purposes. The state does, however, identify those schools that are in 
academic distress based on district performance on standardized tests, dropout and attendance 
rates, teacher quality, professional development hours, and school safety. Three levels of 
academic distress are used (Phases 1, 2, and 3). Also, performance categories are used to rank 
students based on their student achievement scores (advanced, proficient, basic, below basic), 
and these ranks are aggregated to the school and district level. 



Table AR7 

Student Performance Tests, Arkansas 



Test 


Type 


Scoring 


Grades 


Subject Areas 


Notes 


State 


Criterion- 


Scale score, 


4, 6,8 


Math, reading, 


Fourth grade was 


Benchmark 

tests 


referenced 


raw score 




writing 


added in 1999, eighth 
grade in 2000, and 
sixth grade in 2001 


Stanford 


Norm- 


Percentile 


5, 7, 10 


Complete battery: 


Switched from fall to 


Achievement 
Test, 9th 
edition 


referenced 


rank 




reading, math, 
language, social 
science, listening, 
using useful 
information, etc. 


spring testing in 
2002-2003 


End-of-course 


End-of- 




At course 


Algebra, geometry, 


No exit exam is used 


exam 


course 




completion 


literacy 


in Arkansas. 
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Data Usability Issues. Norm-referenced student testing in Arkansas has stayed relatively 
consistent over time and has included multiple grades (5, 7, and 10) for all the years considered 
in this study (1999-2003). The criterion-referenced benchmark tests, however, have been 
introduced step by step between 1999 and 2001. These data allow researchers to track the 
performance of an individual student from year to year. However, a growth score can only be 
calculated for every other year. The content of the criterion-referenced tests has also been 
modified over time. This means that longitudinal analysis of performance of student cohort 
groups can only be performed in a limited way. These data are fairly accessible for researchers in 
aggregate reports. State-, district-, and school-level summaries of student performance are 
published on the Arkansas School Information Site. Individual-level student data, however, are 
not publicly available and must be obtained through special request from the Arkansas 
Department of Education. 
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Table AR8 

Strengths and Challenges of Student Performance Data That Affect Their Use for Policy 
Research, Arkansas 



Criterion 



Strengths 



Challenges 



Availability 

and 

accessibility 



• School- and district-level performance data 
(percentage of students at four proficiency 
levels) are posted on the Arkansas 
Department of Education Web site. 



Completeness • Student performance data can be requested 
from the Arkansas Department of Education 
on all test takers and also on subgroups 
(high-poverty, high-minority, special 

education, limited English proficient). 

Accuracy 



Consistency • Stanford Achievement Test results provide a 
consistent span of test years and grades 
tested. 



Alignment • Demographic and programmatic 

information about test takers can be 
matched with test results. 



• Student level data and disaggregated 
score formats (raw score, scale score) 
must be obtained by special request 
from the Arkansas Department of 
Education. 

• School and district performance data 

cannot be downloaded in a single file 
from the Arkansas Department of 
Education Web site. 

• Test scores are available for a limited 

number of years (grades 4, 6, and 8 
for the state benchmark test; grades 5, 
7, and 10 for the Stanford 
Achievement Test). 

• Data available on the Arkansas 

Department of Education Web site are 
limited to percentage of students at 
four proficiency levels, limiting the 
full range of variability in test results 
for research purposes. 

• State benchmark tests were added 

between 1999 and 2001 (grade 4 in 
1999, grade 8 in 2000, grade 6 in 
2001); test content also was modified 
during these years. 

• Recent addition of grades 6 and 8 state 

benchmark exams limit longitudinal 
comparisons. 



Student, School, and District Characteristics 

Student Characteristics. Arkansas school districts compile information on individual 
students and report these data to the Arkansas Statewide Information System. Most student data 
are reported on six of the seven state reporting periods, starting October 15. The student 
characteristics that are contained in the Arkansas Statewide Information System and of interest 
for this study are listed in Table AR9. 
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Table AR9 

Data Available in State Education Databases on Student Characteristics, Arkansas 



Student 

Characteristic 


Variables Available 


Source 


Unit of 
Analysis 


Socioeconomic 

status 


Free and reduced-price lunch 
program participation 


Arkansas Statewide Information 
System 


Individual 


Race/ethnicity 


Ethnicity 


Arkansas Statewide Information 
System 


Individual 


Grade 


Grade 


Arkansas Statewide Information 
System 


Individual 


Gender 


Gender 


Arkansas Statewide Information 
System 


Individual 


Program 

participation 


• Special education 

• Limited English proficient 
(LEP) 

• Migrant 

• Mobile 
•Title I 


Arkansas Statewide Information 
System 


Individual 



School and District Characteristics. Student characteristics can be aggregated to the 
school and district levels, creating important demographic characteristics. The characteristics of 
interest to policy researchers are listed in Table AR10 with the sources of data for each 
characteristic. Overall, most important characteristics can be viewed and exported from data in 
the Statewide Information System. Characteristics that are measured only at the district level 
include district wealth, which is measured as the sum of residential and nonresidential property 
assessment in the district, and the district tax rate, which is the amount of revenues available 
from local taxes (including residential and nonresidential). 
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Table AR10 

Data Sources for School and District Characteristics, Arkansas 



Characteristic 


Data Sources 






School 


District 


School type 


Arkansas Statewide Information System 


N/A 


School/district size 


Arkansas Statewide Information System 




School level 


Arkansas Statewide Information System 


N/A 


Locale 


U.S. Census Bureau 2000 Census 




Attendance rate 


Arkansas Statewide Information System 




Dropout rate 


Arkansas Statewide Information System 




Per pupil expenditures 


N/A 


Calculation of expenditure 
and enrollment data 


Average class size 


N/A (pupihteacher ratio can be calculated) 




Number of teachers 


Arkansas Statewide Information System 




District wealth 


N/A 


Arkansas Statewide 
Information System 



Community Characteristics. Education databases do not contain information about 
characteristics of the community within which a school and/or district is located. Census 
information provides a number of relevant indicators for community well-being, including 
family income, parent education levels, and receipt of public assistance. 

Summary of Findings 

Arkansas state education data allow investigation of instructional resources and student 
performance. Instructional spending measures include both district-level fiscal expenditures that 
can be broken down to sub function and object-level spending and to individual-level spending 
on staff salaries and benefits. Staff data also include demographic characteristics and 
qualifications such as years of experience and highest degree. Staff data also are coded with 
identifiers that allow researchers to create subgroups of teachers, administrators, and other 
distinct staff categories. The Arkansas Professional Licensure System collects additional data on 
certified staff, including certification information. These monetary and staff measures can be 
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aligned with student performance data. Student performance data include scores from criterion- 
referenced and norm-referenced exams. Including student, school, and district characteristics that 
are collected and housed by state education databases can enhance research using these data. 

Alignment. One feature of state education data in Arkansas is the way in which these data 
are divided so that management of different types of data is conducted by various departments 
within the Arkansas Department of Education. Researchers would need to depend heavily on 
their ability to find mutual identifiers that can bridge the data between two or more databases. 
Alternatively, researchers could work closely with staff at the Arkansas Department of Education 
who are knowledgeable of the data and could merge the data on the variables of interest. Also 
related to the division of education data among different departments in the Arkansas 
Department of Education is the way in which teacher certification data are collected and 
managed. The Professional Licensure Unit collects and maintains certification data that can be 
aligned with staff data from the Arkansas Statewide Information System using a matching 
unique identifier (Social Security number). However, the licensure database is updated 
continuously, without year-to-year archives, so it is much more difficult to align existing staff 
with their certification status. Researchers must rely on accurate issue dates and expiration dates 
of certifications in order to construct the certification information for a given staff person in a 
given year. 

Of greatest concern for researchers regarding the alignment of these data is that financial, 
staff, and student performance data have been improved and therefore changed in the 5 -year 
study period. Financial categories are now attached to individual staff salaries. Beginning in 
2002-2003 staff data were restructured so that individual staff roles were identified and 
recorded, and partial salary information was collected based on the percentage of effort that staff 
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dedicated to different positions. Regarding student performance data, the criterion-referenced 
exam was phased in over a number of years, limiting researchers’ ability to measure longitudinal 
performance on this exam. 

Accessibility. Data access has increased during the study period, and the Arkansas 
Department of Education has developed interactive Web reporting of school data, a file transfer 
protocol (FTP) site that can be used to download school-level staff and student data, and online 
financial reports for all school districts. Arkansas Department of Education staff can respond to 
requests for data by outside users. However, the agency lacks staff dedicated to the role of 
supporting outside data users. Although data managers and staff are responsive to outside data 
requests, these requests must be filled in addition to the regular workload of agency staff and 
often take time and periodic reminders by the requesting party. 

Usability. Instructional spending data collected and organized in Arkansas education 
databases are useful for policy research. Financial expenditures are organized into categories that 
match those used by federal fiscal data reports (Common Core of Data). These data also are 
divided into more refined categories than available at the federal level. For the category of 
instruction, for example, spending can be determined for a range of program areas (regular 
education, special education, etc.). Individual staff salary data are linked to fiscal function 
categories and also provide a measure for employee benefits. Other individual staff data are 
collected in state education databases. These data can be used to identify individuals with 
different staff positions, and a range of demographic and other characteristics can be attributed to 
these individuals. Information about individuals with multiple positions is also collected; these 
data can be linked to partial salary amounts as well. 
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Challenges also exist for researchers using data on resources and student performance. 
Improvements in the data collection of staff salaries and characteristics in 2002 created the 
likelihood that districts did not report their information consistently. Data managers identified a 
few variables related to this change for which districts are not yet reporting accurate data (staff 
full-time equivalency, years of experience). Although staff years of experience data contain 
errors, these data can be used for research purposes; data users may choose to compare multiple 
years of these data to identify and correct obvious errors. The full-time equivalency of staff 
position variable is not usable for research purposes because most of this information is 
incomplete. The completeness and validity of these data may improve over time as districts 
become more familiar with changes in the reporting system. Also, state education databases 
currently do not collect information on class size. An estimate of the pupikteacher ratio can be 
calculated using teacher head counts and student enrollment. However, this calculation cannot 
account for variations in classroom structure, teachers teaching multiple classes, teachers that do 
not work in the classroom, and other sources of miscalculating an accurate pupikteacher ratio. 
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Appendix C 

Louisiana State Education Data 

The Louisiana Department of Education (LDE) is the main source of education data for 
the state of Louisiana. The state has 64 parish school districts and four city school districts, 
employs 60,045 certified staff, including 50,082 classroom teachers, and serves a total of 
734,706 children in the public school system (as of October 1, 2003). 

Overview of Existing State Data 

Researchers examined six education databases housed by the Louisiana Department of 
Education. Four are maintained by the Division of Planning, Analysis, and Information 
Resources(PAIR), one is maintained by the Division of Student Standards and Assessment, and 
the last is maintained by the Division of Teacher Certification and Higher Education. These six 
databases provide information regarding instructional spending, staff characteristics, teacher 
certification, student performance, and student characteristics (see Table LAI). 

The Division of Planning, Analysis, and Information Resources collects and maintains 
data for four databases relevant to this study. This division maintains a central Web site where 
data and reports are available to the public (http://www.doe.state.la.us/lde/pah71419.html). The 
Web site for Data Collection Systems (http://www.doe.state.la.us/lde/pair/638.html) includes 
user’s manuals that define all data elements and posts timelines of data submissions. The four 
Planning, Analysis, and Information Resources databases are described below. 

1 . School districts in Louisiana report district-level financial data to the state via the 
Annual Financial Report (AFR). Data are submitted every fall for the previous school year. 
Similar to Arkansas financial data, fiscal categories match those collected by the U.S. Census 
Bureau on the F-33 form (including revenues by source and expenditures by object, function, and 
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program). The Louisiana Accounting and Uniform Governmental Handbook, available on the 
Planning, Analysis, and Information Resources Web site, describes each category. The Annual 
Financial and Statistical Report summarizes the financial data. This report is published annually 
and posted to the Web site in portable document format (.pdf). Raw data can be downloaded 
from the Web site in Microsoft Access and Excel formats. 

2. The Profile of Educational Personnel (PEP) database houses information about every 
public school staff member, including certified staff such as teachers and principals and 
noncertified staff such as bus drivers. The Profile of Educational Personnel database includes 
information on staff job code(s), salary, education level, years of experience, gender, race, and 
ethnicity. Class codes indicating all the classes taught by a teacher are also reported. District data 
summary tables are downloadable in Microsoft Excel format and published annually in two 
reports: Standard Teacher Counts and Salary Averages and Reported Personnel and District 
Salaries. Budgeted and actual end-of-year figures for staff and salaries (as of October 1) are 
reported. School districts submit Profile of Educational Personnel data to the state every fall and 
spring for the current school year. 

3. The Student Information System (SIS) database includes information about every 
student enrolled in the public school system in Louisiana, including students in grades K-12, 
prekindergarten classes, infant and preschool programs, and nongraded classes. Districts report 
data annually (as of October 1) on each student’s gender, race/ethnicity, free or reduced-price 
lunch program participation, limited English proficient status, special education status, class 
codes, and other variables. School and district summary tables are downloadable in Microsoft 
Excel format and available in portable document format (.pdf) from the Multiple Statistics 
Report on the agency’s Web site. 
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4. The Louisiana Department of Education uses the Annual School Report to verify 
teacher certification, class size, and other measures. Districts submit data to the state every fall 
for the current school year. School and district summary reports are posted annually in the 
District Composite Reports, available on the Planning, Analysis, and Information Resources 
Web site in portable document format (.pdf). 

In addition to the four databases managed by the Planning, Analysis, and Information 
Resources division, two other divisions in the Louisiana Department of Education manage 
student performance and teacher certification data (see Table LAI). 

5. The Division of Student Standards and Assessment manages all student achievement 
data. This division maintains a central Web site 

(http://www.doe.state.la.us/lde/ssa/testhome.html) with summary reports of test data in portable 
document file (.pdf) format. School-, district-, and state-level data are also downloadable in 
Microsoft Excel format. Data reflect results from spring and summer administrations of the 
state’s criterion-referenced and norm-referenced tests. Student test results are also published 
annually in the District Composite Reports. Users may also request student achievement data 
directly from the Louisiana Department of Education. 

6. The Division of Teacher Certification and Higher Education houses the Teacher 
Certification Database (TCR), which contains information about every public school teacher 
certified to teach in Louisiana. Division staff update the database daily as requests for 
certification are received and completed. It includes information about every teacher’s certificate 
type and level, certification area(s) and academic degree(s). The Web site 
(http://www.teachlouisiana.net) has a search tool enabling public users to look up information 
about teachers by name. The District Composite Reports provide teacher certification and 
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qualification information on an annual basis. However, the Web site contains neither 
downloadable raw data nor documentation on data variables. 



Table LAI 

Summary of Existing State Databases, Louisiana 



Data Category 


State Database 


Managing Agency/Department 


Instructional 


Annual Financial Report 


Louisiana Department of Education 


expenditures 


(AFR) 


Division of Planning, Analysis, and 
Information Resources 


Staff characteristics 


Profile of Educational Personnel 
(PEP) 

Annual School Report (ASR) 


Louisiana Department of Education 
Division of Planning, Analysis, and 
Information Resources 


Student 


Student Information System 


Louisiana Department of Education 


characteristics 


(SIS) 

Annual School Report (ASR) 


Division of Planning, Analysis, and 
Information Resources 


Student performance 


Test results database (TST) and 
Web site 


Louisiana Department of Education 
Division of Student Standards and 
Assessment 


Teacher 


Teacher Certification Database 


Louisiana Department of Education 


characteristics 


(TCR) 


Division of Teacher Certification and 
Higher Education 



Instructional Spending 

Money expended for instructional purposes by school districts in Louisiana can be 
examined via two data sources. First, data on each district’s spending in specific instructional 
categories are available from the Annual Financial Report database. School-level expenditures 
are estimated for a limited range of function categories. Second, data on salaries can be obtained 
on individual staff persons from the Profile of Educational Personnel database. These data 
sources are summarized in Table LA2 and described below. 
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Table LA2 

Measures of Instructional Expenditures From District Financial Data, Louisiana 



Fiscal Measure 


Variables 


Description 


Unit of 




Available 




Analysis 


Instruction- 


Instruction 


Activities dealing directly with the interaction 


District 


related expenditure 




between teachers and students 


School 


functions 


Student support 


Activities designed to assess and improve the 


District 




services 


well-being of students and to supplement the 
teaching process 


School 




Instructional 


Activities associated with assisting the 


District 




staff support 


instructional staff with the content and process 


School 




services 


of providing learning experiences for students 






General 


Activities concerned with establishing and 


District 




administration 


administering policy for the district 






School 


Activities concerned with the overall 


District 




administration 


administrative responsibility for a school 






Additional 

functions 




District 


Instruction-related 


Salaries 


Amounts paid to both permanent and temporary 


District 


expenditure 




district employees, including substitutes 




objects 


Benefits 


Amounts paid by the district on behalf of 
employees 


District 




Additional 

objects 




District 


Staff salary data 


Base pay 


The employee’s regular salary 


Individual 




Extra 


Additional salary paid to an employee for 


Individual 




compensation 


additional duties outside his or her regular job 
assignment, such as coaching, yearbook advisor, 
and cheerleader sponsor 






Extended 


The additional salary paid to vocational 


Individual 




employment 

pay 


education instructors 






Professional 


The additional amount of salary provided to an 


Individual 




Improvement 
Program salary 
amount 


employee who is a participant in the Professional 
Improvement Program (PIP) 





Instructional Expenditures. District-level fiscal data submitted via the Annual Financial 
Report include expenditures by function, object, and program. School districts in Louisiana 
maintain highly detailed fiscal data. Flowever, the expenditure data submitted to the state consist 
of less-detailed summary data elements identified by keypunch codes. Each keypunch code 
represents the expenditure amount for a unique combination of function, object, and program 
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codes. For example, keypunch code 0015420 stands for the amount paid under the instruction 
function for salaries for kindergarten teachers in the regular program. 

The fiscal data include expenditures for the following three function categories related to 
instruction: 

1 . The instruction function category is defined as activities dealing directly with the 
interaction between teachers and students. Expenditures in this function category are broken 
down by the following programs: regular education; special education; vocational education; 
other instructional programs such as driver’s education, ROTC, band, and athletics; special 
programs such as bilingual education and prekindergarten; adult/continuing education; and 
community college programs. Within each of these program areas, expenditures are broken down 
into salaries (and several salary subcategories); employee benefits; purchased professional and 
technical services; repairs and maintenance services; tuition; travel expense reimbursements; 
instructional supplies; and equipment. 

2. The pupil support services function is defined as activities designed to assess and 
improve the well-being of students and to supplement the teaching process. Expenditures in this 
function category are broken down into the following areas: child welfare and attendance 
services; guidance services; health services; pupil assessment and appraisal services; and other 
pupil support services. Within each of these areas, expenditures are broken down into salaries 
(and several salary subcategories); purchased professional and technical services; repairs and 
maintenance services; travel expense reimbursements; materials and supplies; and equipment. 
Employee benefits are reported as a total under pupil support services. 

3. The instructional staff support services function category includes activities 
associated with assisting the instructional staff with the content and process of providing learning 
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experiences for students. Expenditures in this function category are broken down into the 
following areas: salaries of directors, supervisors, and coordinators; salaries of clerical and 
secretarial staff; instruction and curriculum development services; travel expense 
reimbursements; instructional staff training services; school library services; other educational 
media services; and other instructional staff services. Within each of these areas, expenditures 
are reported in numerous subcategories. Employee benefits are reported as a total under 
instructional staff services. 

Object-level expenditures are also available in the Annual Financial Report data (salaries, 
benefits, and other objects). Datasets made available by the Louisiana Department of Education 
include total salary expenditures for a wide range of staff positions (e.g., elementary teachers in 
the regular program, principals, superintendents). 

Salaries are reported separately for a variety of staff categories within each program 
under the instruction function. In the regular education program, salaries are reported separately 
for kindergarten teachers, elementary teachers (grades 1-8), secondary teachers (grades 9-12), 
and aides. In the special education program, salaries are broken down for teachers, therapists 
(physical therapy, occupational therapy, speech therapy, etc.), and aides. The gifted and talented 
program includes teachers and aides. The vocational education program includes agriculture 
teachers, home economics teachers, industrial arts teachers, business teachers, other vocation 
education teachers, and aides. Driver education, ROTC, band, athletics, Improving America’s 
Schools Act, bilingual education, prekindergarten, adult/continuing education, and community 
college programs include teachers and aides. All of these programs also report salaries for 
substitute teachers and aides as well as salaries for staff on sabbatical leave. 
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Salaries also are reported separately for a variety of staff categories under the support 
services function and its subfunctions, including pupil support, instructional staff services, 
general administration, and school administration. Under pupil support services, salaries are 
broken down for child welfare and attendance services; guidance services; health services; pupil 
assessment and appraisal services; and other pupil support services. Within these areas, salaries 
are listed separately for supervisors, counselors, nurses, physicians, dental hygienists, 
psychologists, social workers, and clerical staff. Under instructional staff services, salaries are 
listed separately for directors, supervisors, and coordinators; clerical staff; instruction and 
curriculum specialists, staff instructors, library supervisors, librarians, library aides, and 
audiovisual services personnel. Under general administration, salaries are listed separately for 
board of education members, board clerical staff, board legal services staff, superintendents, 
superintendent’s clerical staff, assistant superintendents, assistant superintendent’s clerical staff, 
and other executive administrative staff. Under school administration, salaries are listed 
separately for principals, assistant principals, and clerical staff. 

The Annual Financial Report is the only source of benefits data available in Louisiana. 
Benefits expenditures are reported separately for each program under the instruction function 
(including regular education, special education, vocational education, other instructional 
programs, special programs, adult/continuing education, and community college programs) and 
for each of the support services subfunctions (including pupil support services, instructional staff 
services, general administration, and school administration). For each of these areas, benefits 
expenditures are reported separately for group insurance; Social Security; Medicare; employer’s 
contribution to Louisiana’s teacher retirement system, Louisiana’s school employees’ retirement, 
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and other retirement; unemployment compensation; worker’s compensation; health benefits for 
retirees; sick leave severance pay; and other employee benefits. 

Individual Staff Salary and Benefits Data. School districts submit Profile of Educational 
Personnel data to the Louisiana Department of Education twice a year, including salary amounts 
budgeted for every job position as of October 1 and salary amounts actually paid to every 
employee within the current fiscal year during the June 30 collection. 

The Profile of Educational Personnel database includes four salary amounts for every 
employee: (a) base salary, defined as the employee’s regular salary (including overtime pay for 
the entire year during the June 30 collection); (b) extra compensation, defined as the additional 
salary paid to an employee for additional duties outside of his or her regular job assignment, such 
as coaching, yearbook advisor, and cheerleader sponsor; (c) extended employment 
compensation, defined as the additional salary paid to vocational education instructors; and (d) 
Professional Improvement Program salary supplement, defined as the additional amount of 
salary provided to an employee who is a participant in the Professional Improvement Program. 
No information regarding benefits is collected at the individual staff level. 

Data Usability. The accessibility and quality of data on instructional spending in 
Louisiana support the use of these data for policy research. District-level data from the Annual 
Financial Report and Profile of Educational Personnel are available online. Individual staff data 
from the Profile of Educational Personnel can be requested from the Louisiana Department of 
Education and easily aggregated to school and district levels. Data documentation for Annual 
Financial Report and Profile of Educational Personnel data are also available online and are 
fairly user-friendly. In addition to the raw datasets, the Louisiana Department of Education 
makes several reports and summaries available that are helpful to research audiences. 
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One limitation of these data is the lack of complete school-level data. School-level 
Annual Financial Report data were obtained by special request for the purposes of this study. 
However, these data represent estimates based on individual salary data and district-level fiscal 
data, not actual school expenditures. Also, only one year (2002-2003) and only two function 
categories (no object-level data) are available. Another limitation is the lack of benefits data at 
the individual staff level. Researchers interested in estimating expenditures on individual staff 
benefits must use a calculation of district-level benefits data and individual-level salary data. 
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Table LA3 

Strengths and Challenges of Instructional Expenditure Data That Affect Their Use for Policy 
Research, Louisiana 



Criterion 


Strengths 


Challenges 


Availability 

and 


• Complete Annual Financial Report 
district-level data are available online. 


• Individual- and school-level data 
from the Profile of Educational 


accessibility 


• Documentation for the Annual Financial 


Personnel must be requested from 




Report and Profile of Educational 


the Louisiana Department of 




Personnel data is available and updated 
annually. 


Education. 


Completeness 


• A comprehensive variety of expenditure 
categories is available by function, object, 
and program. 

• Individual salary data are reported as four 
components in the Profile of Educational 
Personnel. 


• Individual-level benefits data are not 
reported. 

• Actual school-level expenditure data 
are not reported (only estimates). 


Accuracy 


• The Louisiana Department of Education 
has multiple automatic mechanisms for 






checking the accuracy of Annual Financial 
Report and Profile of Educational 
Personnel data. 




Consistency 


• Annual Financial Report and Profile of 
Educational Personnel databases have been 


• Due to changes in data on the full- 
time equivalency (FTE) of staff 




mainly consistent over time. 


positions, Profile of Educational 




• Changes in teacher salary calculations are 
well-documented. 


Personnel salary average 
calculations have changed over time. 


Alignment 


• Profile of Educational Personnel salary 






data and Annual Financial Report function 
and object codes are aligned. 





Staff Characteristics 

The Profile of Educational Personnel database contains information about staff 
characteristics. This database includes information collected at the individual level regarding all 
staff, including certified staff (e.g., teachers, administrators, and instructional support staff) and 
uncertified staff (e.g., bus drivers). Data include variables for gender, race/ethnicity, job codes 
with corresponding full-time equivalency (FTE), education level, and years of professional 
experience. For teachers, pilot data for the 2002-2003 school year include all classes taught 
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during that school year. The Teacher Certification Database contains information on all teachers 
and includes data elements such as state certification types, certification areas, degrees, academic 
institutions, national board certification, and teacher test scores (on the NTE, WCET, and 
PRAXIS). Also, data on a teacher’s route to certification (standard or alternative) are available 
from two sources: (a) as part of practitioner license information and (b) with teacher degree 
codes. 



Table LA4 

Staff Characteristics Available in State Databases, Louisiana 



Staff 

Characteristic 


Variables Available 


Unit of 
Analysis 


Source 


Demographics 


Gender and race/ethnicity 


Individual 


Profile of Educational Personnel 


Job code 


Each job assigned to an individual 
(multiple codes possible) 


Individual 


Profile of Educational Personnel 


Full-time 

equivalency 

(FTE) 


Full-time equivalency in each job 
code 


Individual 


Profile of Educational Personnel 


Class codes 


Classes taught (for teachers, pilot 
data 2002-2003) 


Individual 


Profile of Educational Personnel 


Experience 


The years of experience on which a 
salary is based 


Individual 


Profile of Educational Personnel 


Education 


Education level 
Degree (e.g., BA, BS) 
Degree institution 


Individual 

Individual 

Individual 


Profile of Educational Personnel 
Teacher Certification Database 
Teacher Certification Database 


Certification 


Certificate type (i.e., standard, 
nonstandard, and practitioner 
license) 

Certificate level (i.e.. A, B, C) 
Certification area 
National board certification 
Route to certification 


Individual 

Individual 

Individual 

Individual 

Individual 


Teacher Certification Database 

Teacher Certification Database 
Teacher Certification Database 
Teacher Certification Database 
Teacher Certification Database 


Test scores 


Scores on NTE, WCET, or 
PRAXIS 


Individual 


Teacher Certification Database 



Counts and Ratios. Staff full-time equivalency (FTE) counts can be calculated by job 
code, school, district, and demographic characteristics. Researchers also can calculate staff ratios 
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including pupikteacher, pupil: administrator, and teacher: administrator. The Annual School 
Report includes information about class size ranges at the school level, indicating the number 
and percentage of core classes with 1-20 students, 21-26 students, 27-34 students, and more 
than 34 students. 



Table LA5 

Student and Staff Counts and Ratios, Louisiana 



Count/Ratio Measure 


Data Available 


Unit of Analysis 


Source 


Staff full-time 
equivalency (FTE) 
counts 


Can be calculated with 
existing data 


Job code 
School 
District 
Demographics 


Profile of 

Educational 

Personnel 


Class size 


Class size reported by 
school districts 


School 

District 


Annual School 
Report 


Pupihteacher 
Pupifadministrator 
T eacher: administrator 


Can be calculated from 
existing data 


School 

District 


Profile of 

Educational 

Personnel 



Data Usability. State data in Louisiana contain useful information about individual staff, 
with an emphasis on classroom teacher data. Datasets and reports are easily accessible; many are 
downloadable from the agency’s Web site, and annual staffing reports have used consistent 
formats over time. The inclusion of all staff (not simply certified staff) in the Profile of 
Educational Personnel database provides for one central source of staffing data. Two collection 
periods (October 1 and end of year) yield a complete and updated picture of the state’s education 
staff. Users can keep track of multiple job records for any individual staff by using the full-time 
equivalency measures for staff positions. For example, one teacher may be employed at one 
school for 75 percent of a full-time equivalency as a regular classroom teacher and at another 
school (and possibly in another district) for 25 percent of a full-time equivalency as a special 
education teacher. Class codes link teachers to the courses they teach and will link teachers and 
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students starting in 2004-2005. Documentation for the Profile of Educational Personnel database 
is available on the Louisiana Department of Education Web site and is updated annually. 

Other strengths include the fact that the Teacher Certification Database contains all 
certification and education records for the state’s teachers. Teachers with up to seven certificate 
types, up to 16 certification areas, and up to seven degree/institution records were found in the 
Teacher Certification Database. The use of three certification levels (A, B, and C) makes it 
possible to identify new teachers, experienced teachers who have passed the state’s assessment 
program, and experienced teachers with master’s degrees. School-level reports showing the 
number of classes taught by highly qualified teachers integrate data on teacher certification level 
and subject areas and classes taught. Researchers can request individual-level data contained in 
the Profile of Educational Personnel and Teacher Certification Database from the Louisiana 
Department of Education. These individual data can be aggregated to school and district levels. 

Major concerns regarding the Teacher Certification Database are the lack of printed 
documentation and the fact that it is updated continuously with no year-to-year archiving of 
information. Constructing a subset of certified teachers for any given study period is somewhat 
difficult and time-consuming. The annual District Composite Report includes year-to-year 
school-level summaries of teacher certification information, but this information is not consistent 
over time. Specifically, data from the 2000-2001 school year are available on the number and 
percentage of certified teachers, whereas the 2002-2003 report includes number and percentage 
of classes taught by highly qualified teachers, and no information is available for 2001-2002. A 
related challenge concerns the changes in the state’s certification structure. Changes have 
occurred over the last several years, making it difficult to create a coherent picture of the state’s 
teacher certification data. 
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A major challenge regarding the Profile of Educational Personnel database is the 
inconsistent reporting of staff years of experience. State data managers reported that school 
districts interpret the definition of this measure inconsistently, making it unclear whether these 
data reflect staff experience in a position (job code), at the current school or district, or in the 
profession. To reduce inconsistency in these data, the Louisiana Department of Education checks 
the years of experience against certification status, which in part corresponds to years of teaching 
experience. 

Another concern is the use of multiple databases for staff data, increasing the likelihood 
of misaligned data. Staff records are located in the Profile of Educational Personnel database, the 
Teacher Certification Database, and a database containing teacher test scores. Each of these 
databases uses different unique identifiers to organize staff records that must be reconciled 
before data can be merged. Also, the lack of specific grade-level and subject-area identifiers 
makes it difficult to link teachers to the grades and subjects that they teach. Although class codes 
and certification data provide some information on the grade levels and subjects that each teacher 
is teaching (or is certified to teach), this information is only approximate. Additionally, it is 
difficult to link teachers to specific student achievement results, which are reported at the grade 
level and separately for various subject areas (e.g., grade 4 English language arts or grade 8 
mathematics). 
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Table LA6 

Strengths and Challenges of Staff Data That Affect Their Use for Policy Research, Louisiana 



Criterion 


Strength 


Challenge 


Availability 


• Many datasets and reports are available 


• Individual-level data must be 


and 


online. 


requested from the Louisiana 


accessibility 


• The use of anonymous identifiers (as 
opposed to Social Security numbers) 
protects confidentiality. 

• Profile of Educational Personnel User 
Guide is updated annually. 


Department of Education. 

• No printed Teacher Certification 
Database documentation. 

• Online data are not available in 
formats conducive to research use 
(downloadable raw data). 


Completeness 


• Individual-level data are available. 

• Profile of Educational Personnel 
database is a central source of data for 
all staff. 

• All Profile of Educational Personnel 
data are prorated by full-time 
equivalency. 

• Two data collection periods are required 
for the Profile of Educational Personnel 
data. 

• Teacher Certification Database includes 
data on highly qualified teachers and 
national board certification. 

• Separate records are maintained for 
every certificate, certificate area, and 
degree. 


• No year-by-year teacher certification 
data are collected at the individual 
level. 

• Dates are incomplete in the Teacher 
Certification Database (e.g., year of 
degree, issue date, sequence of 
degrees attained). 

• No data are collected on teachers’ 
degree major or coursework. 


Accuracy 


• Class size data are available and more 
accurate than pupil: teacher ratio. 


• Years of experience data may not be 
accurate. 

• Full-time equivalency counts and 
headcounts are identical in school- 
level datasets. 


Consistency 


• Profile of Educational Personnel reports 
have been consistent over several years. 


• Changes have been made over time 
in reporting of certification 
information. 


Alignment 


• Class code variable links teachers to 
classes taught (and will link teachers and 
students in 2004-2005). 


• The use of several personal 
identifiers increases potential for 
misaligned data. 

• Linking teachers to grade levels and 
subjects taught is difficult. 
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Student Performance 

Louisiana has developed a comprehensive system to assess student performance called 
the Louisiana Educational Assessment Program for the 21st Century (LEAP-21). This includes 
criterion-referenced tests in grades 4, 8, and 10 and norm-referenced tests in grades 3, 5, 6, 7, 
and 9. In addition, school-level accountability labels are used based on student achievement test 
results, attendance rates, and dropout rates. 

Criterion-Referenced Tests. Students in Louisiana take the Louisiana Educational 
Assessment Program for the 21 st Century (LEAP-21) tests in the spring of grades 4 and 8. These 
include tests in mathematics, English language arts, science, and social studies. Students take the 
Graduation Exit Exam (GEE-21) for mathematics and English language arts in the spring of 10th 
grade and for science and social studies in the spring of 1 1th grade. 

After an extensive process of test development, the implementation of these tests was a 
gradual process. Starting in the spring of 1999, the Louisiana Educational Assessment Program 
for the 21 st Century (LEAP-21) tests in mathematics and English language arts were given to 
students in grades 4 and 8. The Louisiana Educational Assessment Program for the 2L l Century 
(LEAP-21) tests in science and social studies for grades 4 and 8 were added in the spring of 
2000. The Graduation Exit Exam (GEE-21) for mathematics and English language arts for 10th 
grade were added in the spring of 2001. Finally, the Graduation Exit Exam (GEE-21) for science 
and social studies for 1 1th grade were added in the spring of 2002. 

The criterion-referenced tests in grades 4 and 8 are used for promotion purposes, and 
those students who do not score at the required level are offered intensive summer remediation 
and the opportunity to re-test at the end of the summer. Similarly, the criterion-referenced tests in 
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grades 10 and 1 1 are used to determine eligibility for high school graduation, and those students 
who do not score at the required level are given re-test opportunities in the summer and fall. 

The Louisiana Department of Education Web site provides downloadable files with 
achievement test results as well as reports and data documentation. The information regarding 
the Spring 2003 Louisiana Educational Assessment Program for the 21 st Century (LEAP-21) and 
Graduation Exit Exam (GEE-21) test administrations is the most complete. It includes data files 
with statewide, district-level, and school-level results that indicate the number and percentage of 
students at each of five achievement levels (advanced, mastery, basic, approaching basic, and 
unsatisfactory). Data files include separate results for the initial (spring) test takers, the re-testers, 
and all test takers combined, as well as for the four subject areas. Less complete information is 
available for prior years. 

Norm-Referenced Tests. Students in Louisiana take the Iowa Test of Basic Skills (ITBS) 
in the spring of grades 3, 5, 6, and 7 and the Iowa Test of Educational Development (ITED) in 
the spring of grade 9. These include tests in reading, language, mathematics, science, social 
studies, and sources of information. The test results for 1999, 2000, 2001, and 2002 were based 
on national normative data from 1995. However, the 2003 test results were based on normative 
data from 2000. Therefore, caution must be used when interpreting these data over time. 

The Louisiana Department of Education Web site provides downloadable files with Iowa 
Test of Basic Skills (ITBS) and Iowa Test of Educational Development (ITED) test results as 
well as reports and data documentation. The information regarding the Spring 2003 test 
administration is the most complete. It includes data files with statewide, district-level, and 
school-level results that indicate the composite national percentile rank of the average standard 
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scores. As with the criterion-referenced test data, less complete information is available for prior 
years. 

State Accountability Rankings. Louisiana’s school-based accountability system is 
nationally recognized for its success. Each school in the state is rated every fall based on the 
school’s performance the previous school year. The rankings are based on achievement test 
results (60 percent criterion-referenced tests and 30 percent norm-referenced tests) as well as on 
attendance and dropout rates (10 percent). 



Table LA7 

Student Performance Tests, Louisiana 



Test 


Type 


Scoring 


Grades 


Subject Areas 


Louisiana 
Educational 
Assessment 
Program for the 
2 1 st Century 


Criterion- 

referenced 

test 


Raw score 
Scale score 

Percentage of students at 
each of five achievement 
levels 


4 and 8 


English language arts 
Mathematics 
Science 
Social studies 


Graduation Exit 
Exam 


Criterion- 

referenced 

test 


Raw score 
Scale score 

Percentage of students at 
each of five achievement 
levels 


10 and 11 


English language arts ( 1 0) 
Mathematics ( 1 0) 

Science (11) 

Social studies (11) 


Iowa Test of 
Basic Skills 

Iowa Test of 

Educational 

Development 


Norm- 

referenced 

test 


Composite national 
percentile rank of the 
average standard scores 


3, 5, 6, 7, and 
9 


Reading 

Language 

Mathematics 

Science 

Social studies 

Sources of information 



Data Usability. Student performance data for Louisiana are very accessible. Data are 
easily downloadable from the agency’s Web site. Reports and data documentation are also 
available online. Individual student-level data, however, must be obtained by special request to 
the Louisiana Department of Education. 
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Researchers must be aware of the challenges concerning the use of criterion-referenced 
test scores. A major challenge of these test data is the lack of consistent testing over time. As 
discussed above, the criterion-referenced tests were phased in gradually, limiting the capacity for 
any analysis over time. Results are available for fourth- and eighth-grade English language arts 
and mathematics for 1999 through 2003. However, results for grades 4 and 8 in science and 
social studies only go back to 2000. Results for high school English language arts and 
mathematics go back to 2001, whereas high school science and social studies go back only to 
2002. According to data documentation ( Annual Louisiana Educational Assessment Program for 
the 21st Century and Graduation Exit Exam Report from Spring 2003), the difficulty of the tests 
remains the same from year to year through the use of a process known as test equating that 
involves scaled scores (computed from raw scores). However, data managers cautioned against 
any comparisons across years and across grades, which obviously represents a challenge to 
policymakers and researchers interested in longitudinal analysis. 

The way that the test results are reported on the Louisiana Department of Education Web 
site also has changed over time. As mentioned above, the spring 2003 information is the most 
complete. Prior to this year, data files on the Web site did not include separate results for initial 
test takers, re-testers, and all test takers combined. Data files also did not indicate the number of 
students at each achievement level (that is, in some cases only the percentage of students at each 
level is listed). Furthermore, it is not always clear whether the results are for all test takers, initial 
test takers, or re-testers and if they represent number of students or percentages of students. 
Finally, the spring 1999 Louisiana Educational Assessment Program for the 21 1,1 Century data are 
available only at state and district levels (not school level). These challenges are surmountable, 
however. School-level analyses can be performed with data from the spring 2000 through spring 
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2003 tests, and conversations with data managers can clear up any confusion about the reporting 
of the results. 

A major strength of the data from the norm-referenced tests is the capacity for analysis 
over time. The test administrations of the Iowa Test of Basic Skills (ITBS) and Iowa Test of 
Educational Development (ITED) have remained consistent from spring 1999 through spring 
2002. A challenge concerns the spring 2003 test administration that yielded test results based on 
normative data from 2000 (a change from the earlier 1995 norms). Another challenge is the lack 
of detail in the test data available online because only composite scores are reported (no specific 
subject-area data). Finally, only statewide and district-level data are available online from the 
1999 test administration; school-level data must be requested from the Louisiana Department of 
Education. 

Two major strengths of the state accountability ratings are the reporting of adequate 
yearly progress (AYP) and the recent addition of the subgroups reports. The most obvious 
concerns with the system relate to the many changes since its inception in 1998-1999, the most 
recent being changes in the definitions of the performance labels (2002-2003) and of the growth 
labels (2003-2004). 



Southwest Educational Development Laboratory 



113 




Investigation of Education Databases in Four States to Support Policy Research on Resource Allocation 



Table LA8 

Strengths and Challenges of Student Performance Data That Affect Their Use for Policy 
Research, Louisiana 



Criterion 



Strengths 



Challenges 



Availability • School- and district-level 

and performance data (percentage of 

accessibility students at five proficiency levels) 

are downloadable from the Louisiana 
Department of Education Web site. 

• Other score formats (raw score and 
scale score) can be requested from 
the Louisiana Department of 

Education. 

Completeness • Student performance data are 

available on all test takers and on 
subgroups (free and reduced-price 
lunch program participants, 
race/ethnic groups, special education, 
and limited English proficient 

students). 

Accuracy 



Consistency • Grades tested for Iowa Test of Basic 
Skills, Iowa Test of Educational 
Development, and Louisiana 
Educational Assessment Program for 
the 21st Century, in combination, 
provide a consistent span of grade 
levels. 



Alignment • Demographic and programmatic 

information about test takers are in 
some cases reported with test results. 



• Student-level data are not publicly 
available and must be obtained by special 
request to the Louisiana Department of 
Education. 



• Criterion-referenced test scores are 
available for a limited number of years 
(i.e., grades 4 and 8 Louisiana 
Educational Assessment Program for the 
21st Century and grade 10 Graduation 
Exit Exam). 



• Criterion-referenced test data available 

on the Louisiana Department of 
Education Web site are limited to 
percentage of students at five 
achievement levels, limiting the full 
range of variability in test results for 
research purposes. 

• Criterion-referenced test measures only 
go back a few years in some cases. 

• In 2003, the state added a pre-GED Iowa 
Test of Basic Skills. 

• Iowa Test of Basic Skills scores reflect 
1995 nonns until 2003, when the new 
2000 nonns were applied. 

• Accountability labels change over time. 

• Due to changes in norming standards for 
the norm-referenced test and 
incompatible scaling for the criterion- 
referenced tests (grades 4, 8, and 10), 
possibility for longitudinal analysis is 
limited. 
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Student, School, and District Characteristics 

Student Characteristics. School districts submit data to the state on the characteristics of 
individual students via the Student Information System. Data elements include each student’s 
gender, race/ethnicity, free and reduced-price lunch program participation, special education 
status, limited English proficiency (LEP) status, grade level, and other characteristics. Many 
important school and district characteristics can be derived from the student-level Student 
Information System data. The Louisiana Department of Education publishes numerous files on 
its Web site with aggregates of these data (the Multi- Stats Reports). Researchers also can request 
student-level and grade-level data from the Louisiana Department of Education. 
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Table LA9 

Data Available in State Education Databases on Student Characteristics, Louisiana 



Student Characteristic 




Source 






Individual 


Grade 


School 


District 


Gender 


By request 
from the 
Louisiana 
Department of 
Education 


By request 
from the 
Louisiana 
Department of 
Education 


Multi- Stats 
Report 


Multi- Stats 
Report 


Race/ethnicity 


By request 
from the 
Louisiana 
Department of 
Education 


By request 
from the 
Louisiana 
Department of 
Education 


Multi- Stats 
Report 


Multi- Stats 
Report 


Special education status 


By request 
from the 
Louisiana 
Department of 
Education 


By request 
from the 
Louisiana 
Department of 
Education 


Multi- Stats 
Report 


Multi- Stats 
Report 


Limited English proficient 
(LEP) status 


By request 
from the 
Louisiana 
Department of 
Education 


By request 
from the 
Louisiana 
Department of 
Education 


Multi- Stats 
Report 


Multi- Stats 
Report 


Free and reduced-price 
lunch program 
participation 


By request 
from the 
Louisiana 
Department of 
Education 


Multi- Stats 
Report 


Multi-Stats 

Report 


Multi- Stats 
Report 


Grade enrollment 


By request 
from the 
Louisiana 
Department of 
Education 


Multi- Stats 
Report 


Multi- Stats 
Report 


Multi- Stats 
Report 



School Characteristics. In addition to the student characteristics data that can be 
aggregated to the school level, several variables are available to measure school characteristics. 
These include type of school (elementary, middle/junior high, high, and combination school) and 
grades enrolled (e.g., kindergarten only, grades 2-5, grades 6-8). Four different student counts 
are available: K-12 student count, all elementary and secondary student count, students funded 
under the state’s funding formula, and all reported students. 
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District Characteristics. In addition to the student- and school-level characteristics that 
can be aggregated to the district level, district wealth measures are available. Two commonly 
used indicators include the local wealth factor and fiscal capacity per pupil. The school and 
district characteristics of interest to researchers are available from several databases, including 
the Annual School Report and the Annual Financial Report (discussed above), the Multi-Stats 
Reports, the Reported Salaries and Personnel, the Minimum Foundation Program Funding 
Formula Accountability Report, and online accountability reports. 



Table LA 10 

Data Sources for School and District Characteristics, Louisiana 



Characteristics 




Data Sources 




School 




District 


School type 


Annual School Report 




N/A 


Grade range 


Annual School Report 




Annual School Report 


Total enrollment 


Multi-Stats Report 




Multi- Stats Report 


Class size 


Annual School Report 




Annual School Report 


Number of teachers 


Reported Salaries and Personnel 


Reported Salaries and Personnel 


Per pupil expenditures 


MFP Funding Formula 
Accountability Report 




Annual Financial Report 


Accountability data 


Online accountability reports 


Online accountability reports 


District wealth 


N/A 




Annual Financial Report 



Community Characteristics. Education databases do not contain information about the 
characteristics of the communities within which schools or districts are located. Census 
information provides a number of relevant indicators for community well-being, including 
family income, parent education levels, and receipt of public assistance. 

Summary of Findings 

Data collected and managed by the Louisiana Department of Education provide useful 
information for resource allocation policy research. At the same time, improvements in ways that 
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data are collected, reported, and made available to data users could enhance the capacity of these 
data. 

Accessibility. Most of the education data in Louisiana are accessible through the central 
Data and Reports section of the Louisiana Department of Education Web site. Data are available 
online on instructional spending; staff characteristics; student performance; and student, school, 
and district characteristics. Much information is available at state and district levels, although 
some is also available at the school level. Student characteristics and achievement test results are 
reported at the grade level. Additional data can be requested easily and obtained in a timely 
manner from the Louisiana Department of Education, whose staff and data managers are both 
knowledgeable and helpful. 

The accessibility of these data would be improved if additional datasets were made 
available online (e.g., school-level fiscal and staffing data; individual-level student and staff data 
with anonymous identifiers that are consistent across datasets; raw data in Microsoft Access or 
SAS formats; and raw and scale score formats of student achievement data). The accessibility of 
these data would also be enhanced with improved data documentation that is posted on the same 
Web site. Improved documentation of student achievement data and teacher certification and 
education data would be especially helpful. 

Alignment. The consistent use of numeric district- and school-level identifiers makes it 
possible to aggregate individual student and staff data to these levels. Individual-level student 
and staff data also make it possible to specify which individuals should be included in a given 
aggregate (e.g., all classroom teachers or only classroom teachers in the regular program in a 
given school or district). 
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Regarding the alignment of various types of staffing data, the use of several different 
identifiers is somewhat problematic. When collecting data on teacher salary, experience, 
certification, education, and test scores, researchers originally obtained separate data files, which 
included four different identifiers. A subsequent request to the Louisiana Department of 
Education yielded data files with only two different identifiers, but the potential of misaligned 
data was not eliminated. Obviously, the usability of these data would be improved by the use of a 
single identifier. 

The possibility of linking teachers to the grades and subjects that they teach (and thus to 
student achievement results, which are reported separately by grade level and subject area) is 
uncertain but may possibly be achieved through the use of class codes and/or certification area 
data. Finally, the education data have the capacity to link teachers to the actual classes they teach 
during a given school year. For the 2002-2003 school year, data also were collected on the 
number of students in each class taught. Data collection is currently being piloted to link students 
to classes (which will allow the linking of teachers and students) and is projected to be fully in 
effect starting with the 2004-2005 school year. 

Usability. The education data from the Louisiana Department of Education have high 
usability for policy research because they are generally complete, accurate, and consistent. In 
terms of completeness, data are available at a great level of detail and without overwhelming 
complexity. Much information is collected at the individual student and staff levels. The 
sophisticated staffing system allows for the prorating of full-time equivalents (FTEs) and salaries 
through one central database that includes all education staff (not simply certified staff or 
instructional staff); student achievement results can be reported along with the demographics of 
the test takers and for various student subgroups. Additional indicators of completeness include 
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class size data, the class codes that soon will link teachers and students, and the teacher 
certification database with separate records for each of a given teacher’s certificates and 
academic degrees (as opposed to current certificate or highest degree). 

More detailed data would nevertheless enhance usability. Lower levels of data would 
improve the capacity for research use. For example, actual school-level expenditures (as opposed 
to estimates) and individual staff data on benefits would provide important detail to financial 
data. Additional data that would be useful for research purposes include each teacher’s degree 
major and coursework. Criterion-referenced student achievement data for additional grade levels 
also would greatly enhance the potential for research; the Louisiana Department of Education is 
currently developing test items for new criterion-referenced tests to be added to the current tests 
for grades 4, 8, and 10. 

In terms of accuracy, examinations of these data along with conversations with data 
managers showed that the education data are generally valid, accurate representations of what 
they are intended to measure. Among the few exceptions are data on staff years of professional 
experience, which are problematic due to inconsistent reporting by districts. Data accuracy might 
of course be improved with better instructions on what type of experience to report (years of 
experience in the current job assignment, at the school, in the district, or in the profession). 
Examinations of school-level staffing aggregates revealed no differences between the full-time 
equivalency count data and the headcount data, which also may indicate inaccurate reporting. 

Finally, in terms of consistency, most of the Louisiana Department of Education data 
have been fairly consistent during the period of years examined for this study. However, teacher 
certification data and reports have undergone certain changes due to shifts in the certification 
structure and requirements, and data on teacher test scores have been inconsistent due to changes 
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in the test used to assess teacher skills and knowledge. Even more importantly, the 
implementation of new criterion-referenced student achievement tests starting in 1998-1999 
limits the capacity for certain analyses over time; the worst case is high school science and social 
studies because the new test was not implemented until the spring of 2002. The possibility may 
exist for conducting valid school-level analyses of improvements in student achievement by 
comparing fourth-grade achievement for several consecutive years, even though the process of 
test equating (which is supposed to make the test equally difficult from year to year) is imperfect. 
However, the ability of researchers to conduct any longitudinal cohort analysis — where 
individual students are tracked over time — depends on having student-level data as well as 
comparable test scaling (e.g., from the fourth-grade to the eighth-grade criterion-reference tests). 
Cohort analysis may be conducted with the norm-referenced student achievement data up until 
the spring of 2003, when the national norming group changed. However, policymakers tend to 
view the norm-referenced test as less relevant than the criterion-referenced test, making a 
longitudinal analysis of criterion-referenced scores the preferred choice. 
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Appendix D 

New Mexico State Education Data 

The New Mexico Public Education Department is the primary source of education data 
for the state. The department oversees the provision of education services in 750 schools 
organized into 89 districts. Information is collected on financial activity in each school district, 
on the characteristics of more than 20,000 instructional staff, and on the performance and 
demographics of over 300,000 students per year. Data collection and management are guided 
largely by the reporting requirements of state and federally funded programs, data needs of the 
state’s accountability system and teacher accreditation system, and the state’s financial reporting 
and accountability mechanisms. 

Overview of Existing State Data 

For the purposes of this study, researchers examined the following four major databases 
in New Mexico that provide information about instructional expenditures, staff and student 
characteristics, teacher certification, and student performance (see Table NM1). 

1 . Financial information about New Mexico school districts is collected and managed by 
the School Finance Unit at the New Mexico Public Education Department. The New Mexico 
Public Education Department collects financial information from all districts and charter schools 
in the state and organizes these data into multiple categories, including expenditures, revenues, 
estimated budgets, cash balances, and capital improvements and debt. Instructional expenditures, 
of specific interest to this study, are organized into a chart of accounts with a standard set of 
funds, functions, and objects. Data are compiled into one document, New Mexico Public School 
Finance Statistics, and downloadable from the New Mexico Public Education Department Web 
site (http://www.ped.state.nm.us/div/fin/school.budget/nm.stat.03/index.html). Definitions of 
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fiscal variables are recorded in the agency’s document Supplements to the Manual of Procedures 
and may be requested from the School Finance Unit staff. 

2. The Accountability Data System (ADS) is the statewide student and staff information 
system for K-12 public education in New Mexico. School districts are asked to collect and 
submit detailed student and staff information for inclusion in the Accountability Data System. 
Data are collected on all public school students enrolled one-half time or more and on all 
certified and noncertified staff (including short- and long-term substitutes). Districts are required 
to submit student and staff data for three reporting periods ending on the 40th, 80th, and 120th 
days of the school year. Although summaries of information collected through the 
Accountability Data System are posted to the New Mexico Public Education Department Web 
site (http://www.ped.state.nm.us/div/ais/data/dcrfactsheets.html), data files must be requested 
from agency staff. Accountability Data System variables are defined in the agency’s document 
Accountability Data System Information Manual, which is also available by request from the 
agency’s Chief Information Office. 

3. The Educator Quality Support Unit at the New Mexico Public Education Department 
collects and manages a licensure database that contains certification information about each 
individual teacher in the state. These data result from application information and teacher test 
scores that are submitted to the unit as part of the teacher licensure process. Staff upload data 
from the licensure database to the Accountability Data System on a regular basis to allow for 
wider agency use of these data. Data must be requested from the Education Quality Support 
Unit. Data documentation and variable definitions, however, are not recorded on any printed or 
electronic source. 
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4. The Accountability and Assessment Unit at the New Mexico Public Education 
Department manages and reports on the state’s student performance results. The unit collects 
multiple measures of student performance, including a norm-referenced student achievement 
test, a high school competency exam, a state writing assessment, and a Spanish-language 
achievement test. Data are compiled on all of these tests for all students that take the exam(s). 
Student performance data may be requested from the agency’s Accountability and Assessment 
Unit or the Chief Information Office. Data documentation and variable definitions, however, are 
not recorded on any printed or electronic source. 



Table NM1 

Summary of Existing State Databases, New Mexico 



Data Category 


State Database 


Managing Agency/Department 


Instructional 

expenditures 


New Mexico Public School 
Finance Statistics 


New Mexico Public Education Department, 
School Finance Unit 


Staff characteristics 


Accountability Data System 


New Mexico Public Education Department, 
Accountability and Assessment Unit 


Student characteristics 


Accountability Data System 


New Mexico Public Education Department, 
Accountability and Assessment Unit 


Student performance 


Student performance test 
results 


New Mexico Public Education Department, 
Accountability and Assessment Unit 


Teacher licensure 


Licensure Database 


New Mexico Public Education Department, 
Educator Quality Support Unit 



Instructional Spending 

Dollars spent for instructional purposes by school districts in New Mexico can be 
measured through two data sources. First, researchers can obtain financial data on each school 
district on spending in specific instructional areas. Second, researchers can request salary 
information on individual instructional staff persons from the Accountability Data System. These 
data sources are summarized in Table NM2 and described below. 
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Table NM2 

Measures of Instructional Expenditures From District Financial Data, New Mexico 



Fiscal Measure 


Variables 

Available 


Description 


Unit of 
Analysis 


Instruction-related 

expenditure 

functions 3 


Direct instruction 


Expenditures related to direct delivery of 
instruction, including expenses for teachers, 
substitutes, and instructional assistants 


District 




Instructional 

support 


Expenditures related to support of instruction 
(expenses for principals, instructional 
coordinators, and student support staff) 


District 




Administration 


Expenditures related to administration of the 
district, including expenses for superintendent, 
administrative staff, and board members 


District 


Instruction-related 
expenditure objects 3 


Personnel services 


Expenditures for compensation (not including 
benefits) paid to staff 


District 


Employee benefits 


Expenditures for benefits provided to staff 
(insurance, retirement, worker’s comp, etc.) 


District 




Purchases services 


Expenditures for purchases (contracted services, 
rents and leases, maintenance and repair, etc.) 


District 




Supplies and 
materials 


Expenditures for materials such as 
textbooks/software and general supplies 


District 




Travel and training 


Expenditures for travel and training for staff, 
board members, parents, and/or students 


District 




Capital outlay 


Expenditures for fixed ($ 1 ,000 and over) and 
supply (under $1,000) assets 


District 


Staff salary data b 


Salary 


• Annual base salary rounded to the nearest 
dollar (does not include extra service pay) 

• Partial salaries for staff with multiple roles 
are not reported but can be estimated by using 
full-time equivalency (FTE) count data 


Individual 

classified 

staff 




Source of salary 


Percent of base salary paid from 

• State operational funds 

• Federal funds 

• Medicaid in the schools funds 

• Private funds 

• Public funds other than state operational 


Individual 

classified 

staff 



“These data are contained in the Chart of Accounts. 
b These data are contained in the Accountability Data System. 



Instructional Expenditures. Two function categories collected by school districts in New 
Mexico pertain to the expenditures dedicated for instruction: (a) direct instruction and (b) 
instructional support (see Table NM2). Direct instruction includes payroll spending (personnel 
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services and employee benefits) for teachers, substitutes, and instructional assistants. It also 
includes spending within the other object categories under this function, including purchased 
services, supplies and materials, travel and training, and capital outlay (fixed and supply assets) 
that pertain to classroom instruction. Instructional support includes payroll spending for staff 
who directly or indirectly support instruction of students (e.g., principals, instructional 
coordinators, student support staff). It also includes spending in the object categories mentioned 
above that relate to instructional support activities. An additional function category that is 
indirectly related to instructional spending but important for comparative purposes is 
administration. This categoiy contains spending for central administration, including the costs of 
the superintendent and administrative support. 

Individual Staff Salary Data. Salaries for instructional and administrative staff can be 
determined using the broad fiscal categories described in the financial data above. For individual 
teachers, instructional support staff, student support staff, and administrative staff, salary data are 
collected in the Accountability Data System. Each staff record contains a base salary amount as 
well as data on the source(s) of revenue that funded the base salary. 

Data Usability. Researchers considered data on instructional spending from both the 
financial database and the Accountability Data System for their potential usefulness for policy 
research purposes (see Table NM3). Overall, the district-level data on instructional expenditures 
and the individual-level staff salary data would provide useful data sources for conducting 
resource allocation research. These data are available via two sources, one providing a district- 
level perspective of overall spending in instruction and the other providing detailed information 
about salaries that could be aggregated to grade or classroom salary measures. Data on district 
spending for instructional functions and objects are downloadable from the New Mexico Public 
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Education Department Web site, and detailed documentation on these data is available by request 
from the School Finance Unit. According to data managers, database contents and data 
definitions for instructional expenditures in the financial database have not changed 
substantively during the years considered for this study (1999-2003). The individual salary data 
from the Accountability Data System also have been collected in a consistent manner from 1999 
to 2003. The individual salaries also can be aligned directly to staff characteristics that are also 
located in the Accountability Data System. This allows researchers to use staff information on 
full-time equivalency of different positions to estimate, for example, how much of an 
individual’s salary is dedicated to instructional or noninstructional positions. 

A significant amount of data on instructional spending are collected and reported in state 
education databases and could, in theory, be used in policy research. The following four 
challenges that researchers face in this application are apparent. 

1 . Access to data from the Accountability Data System is restricted and requires a special 
request to the New Mexico Public Education Department. 

2. Although data on spending on instructional function and object categories are available 
at the state and district levels, similar information is not collected by the agency for school-level 
spending. 

3. The individual-level salary data that are provided via the Accountability Data System 
are not consistent with the district-level fiscal salary data. The Accountability Data System 
provides the base salary being paid to each staff person reported at the three reporting dates 
(40th, 80th, 120th days) and also labels the district at which the staff person works; however, an 
aggregation of all salaries for one district would not align to the actual amount expended for 
salaries at that district as recorded in the financial database. This is because the number of days a 
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staff person is employed is not tracked. Additionally, definitions differ between the base salary in 
the Accountability Data System and personnel services object in the financial database. 

4. Finally, spending for staff benefits is tracked in the fiscal database and broken down 
into subcategories (educational retirement, retiree health care, health/medical, life, dental, etc.). 
However, since benefits spending data are available only at the district level, researchers cannot 
determine the cost of benefits or extra duty assignments for individual staff within a district. 

Thus, allocation of benefits for individual employees must be calculated as a proportion of all 
district spending on benefits rather than as an actual cost for each employee. 
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Table NM3 

Strengths and Challenges of Instructional Expenditure Data That Affect Their Use for Policy 
Research, New Mexico 



Criterion 


Strengths 


Challenges 


Availability 


• Data are available from two sources: 


• Access to staff salary data is restricted 


and 


district expenditures and individual staff 


and requires special request to the New 


accessibility 


salaries. 

• District financial data are downloadable 
in Microsoft Excel spreadsheets from the 
New Mexico Public Education 
Department Web site. 

• Data documentation is updated annually 
and available by request from the New 
Mexico Public Education Department. 


Mexico Public Education Department. 
• School level expenditure data are not 
collected. 


Completeness 


• District financial data are available for all 
districts and all years of the study period. 

• Staff salary data are collected for 
certified and non-certified staff for all 
schools and all years of the study period. 


• Some data collection problems persist, 
especially for districts that encounter 
changes in leadership or accounting 
staff. 


Accuracy 


• Data managers report that consistency of 
the financial data collection process has 
increased the accuracy of district data 
reports. 

• Automatic error checking is conducted 
before the districts formally submit data 
to the New Mexico Public Education 
Department. 


• Some data collection problems persist, 
especially for districts that encounter 
changes in leadership or accounting 
staff. 


Consistency 


• No substantive changes were made in 
data collection procedures or variable 
definitions during the study period. 


• Different reporting procedures for staff 
salary data and district salary 
expenditure objects create 
noncomparable data. 


Alignment 


• Individual staff salary data can be aligned 
with other staff characteristics. 

• Staff salary data can be averaged across 
grades, schools, and districts. 


• Staff benefits are collected at the district 
level and not at the individual level. 

• Individual-level expenditures for 
benefits can only be estimated by 
averaging across all staff in a district. 



Staff Characteristics 

Information about staff is collected at the individual level in the Accountability Data 
System. Data measures exist for demographic characteristics of staff, information about the 
classroom assignment of teachers and aides, and full-time equivalents of particular staff positions 
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in schools and districts. Additionally, the Education Quality Support Unit of the New Mexico 
Public Education Department collects teacher certification information, which is regularly 
uploaded to the Accountability Data System database (see Table NM4). The Accountability Data 
System database also contains individual-level staff characteristics collected at the 40th, 80th, 
and 120th days of each school year. Certification data and teacher test scores are collected and 
maintained separately from the Accountability Data System; however, contents of the database 
are regularly uploaded to this data system. One feature of the licensure database is that it is 
continually updated without archiving, so no year-to-year data exist. Further limiting the 
usability of this data for policy research is the lack of issue date and end date information for 
certification. 



Table NM4 

Staff Characteristics Available in State Databases, New Mexico 



Staff 


Variables Available 


Unit of 


Source 


Characteristic 




Analysis 




Position and full- 
time equivalency 
count 


Position of staff person and full-time 
equivalency devoted to each position 
the individual fills 


Individual 


Accountability Data System 


Experience 


Years of experience in the district 


Individual 


Accountability Data System 




Years of experience out of district 


Individual 


Accountability Data System 


Education 


Bachelor’s degree institution 


Individual 


Accountability Data System 




Highest degree earned 


Individual 


Accountability Data System 




Highest degree institution 


Individual 


Accountability Data System 


Demographics 


Gender 


Individual 


Accountability Data System 




Race/ethnicity 


Individual 


Accountability Data System 




Birth date 


Individual 


Accountability Data System 


Certification 


Subject area of license 


Individual 


Licensure Database 


(teachers) 


Grade level of license 


Individual 


Licensure Database 




Type of license (standard, waiver, 
emergency, etc.) 


Individual 


Licensure Database 




Level of license (1-3 tiers) 


Individual 


Licensure Database 




Raw score on NM teacher assessment 


Individual 


Licensure Database 
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Counts and Ratios. Staff counts and ratios can be calculated from individual data in the 
Accountability Data System (see Table NM5). Staff full-time equivalency (FTE) counts can be 
created by position, school, district, or demographic characteristics. Researchers can calculate a 
pupihteacher ratio using student enrollment information and an aggregate of classroom teachers 
listed in the Accountability Data System. State data do not contain actual class size information. 
However, course information, including the student identifier, teacher identifier, and course 
number, are collected in the Accountability Data System; researchers might potentially be able to 
estimate class size using this information. Other staff ratios such as teacher: administrator or 
pupil: administrator also can be calculated using aggregated staff full-time equivalency (FTE) 
counts from the Accountability Data System. 



Table NM5 

Student and Staff Counts and Ratios, New Mexico 



Count/Ratio Measure 


Data Available 


Unit of 
Analysis 


Source 


Student enrollment 


Can be calculated from 
existing data 


Grade 

Program 

School 

District 

Demographic 

subgroups 


Accountability Data 
System 


Staff full-time 


Can be calculated from 


Job position 


Accountability Data 


equivalency counts 


existing data 


School 

District 

Demographic 

subgroups 


System 


Class size 


Possibility of calculating 
actual class size from 
existing data 


Class 


Accountability Data 
System 


Other ratios: 


Can be calculated from 


School 


Accountability Data 


• Pupihteacher 

• Pupihadministrator 

• Teachenadministrator 


existing data 


District 


System 
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Data Usability. As summarized in Table NM6, data on staff characteristics are useful in 
many ways for policy research purposes and also contain some limitations for these purposes. 
These data are collected on an individual level, creating flexibility for researchers to create 
aggregated data on a number of subgroup categories such as grade, school, and demographic 
characteristics and to create student and staff ratios for schools and districts. The structure of the 
database includes a way to link individual students to their teachers for each class in which they 
are enrolled. The Accountability Data System has been in existence for many years and has 
established automated data collection, validation, and error correction mechanisms. Since staff 
data contain job code information, teacher data can be isolated for research purposes. 

The most important data usability issue concerns accessibility of individual-level staff 
data for use by outside researchers. State and district summaries of staff information are 
available on the New Mexico Public Education Department Web site; however, these data are 
not downloadable and postings are not consistent with respect to content and timeliness. Also, no 
established mechanism exists for sharing individual-level data with outside users, and data 
managers struggle with federal Family Educational Rights and Privacy Act (FERPA) 
requirements because the unique identifier for staff in the Accountability Data System database 
is the individual’s Social Security number. 

Another concern regarding data usability is the fact that teacher certification data are 
compiled from a different source using different procedures than the Accountability Data 
System. Data on the same individuals originate from two streams of data collection using 
different procedures, which creates greater possibility of misaligned data on teachers. 
Additionally, certification data are collected using a cumulative process in which updates are 
made to teacher information without annual archives. Because this database does not contain the 
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issue date or end date of teacher certificates, researchers face a significant barrier to conducting 
analysis on this variable. 



Table NM6 

Strengths and Challenges of Staff Data That Affect Their Use for Policy Research, New Mexico 



Criterion 



Strengths 



Challenges 



Availability • Individual-level staff characteristics are 
and collected with the ability to aggregate 

accessibility data up to grade, school, or district levels. 

• Data documentation is updated annually 
and available by request from the New 

Mexico Public Education Department. 

Completeness • Three data collection periods ensure that 
most students and staff are accounted for 
given partial-year staff contracts and 
within-year staff and student turnover. 

• Staff data are collected for certified and 
noncertified staff for all schools and all 

years of the study period. 

Accuracy • Automatic error checking is conducted 
before districts formally submit data to 
the New Mexico Public Education 

Department. 

Consistency • No substantive changes were made in 
data collection procedures or variable 
definitions during the study period. 

• Data collection documentation provides 
clear instructions with helpful examples 

for the user. 

Alignment • Staff characteristics can be linked to 
salary data. 

• Individual students can be linked to their 
teachers for each class in which they are 
enrolled. 

• Teacher certification data are regularly 
uploaded to the main staff database. 



• Staff data are not publicly available and 
must be obtained by special request 
from the New Mexico Public Education 
Department staff. 



• Teacher certification data lack critical 
information about dates for which 
certifications are valid. 



• Accuracy of certain variables of interest 
are suspect (e.g., staff years of 
experience, student poverty). 



• Teacher data are a combination of two 
data collection efforts, staff 
characteristics and teacher certification, 
creating the possibility of misalignment 
if data are merged. 

• The teacher certification database is 
updated cumulatively without year-to- 
year archive information for aligning 
with other staff data. 
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Student Performance 

The state of New Mexico uses multiple student achievement tests for measuring 
performance: norm-referenced tests in English and Spanish, a 10th- grade competency exam, and 
a state writing assessment. Student performance is also gauged at the school level using school 
accountability rankings that are based primarily on test scores. 

Norm- and Criterion-Referenced Tests. New Mexico has traditionally relied on norm- 
referenced student achievement tests to track performance of students. The CTBS/TerraNova 
Survey Plus was used through 2001-2002 for grades 3-9. The state reported median percentiles 
for math, reading, language arts, total score, science, and social studies. The state also reported 
scale scores for criterion-referenced questions that were identified within the norm-referenced 
exams in math, language arts, science, and social studies. The CTBS/Terra Nova Survey Plus 
was replaced in 2002 by the Terra Nova 2 nd edition Complete Battery (CAT) for grades 3-9. The 
state reported median percentiles for math, reading, language arts, total score, science, and social 
studies. A true criterion-referenced exam was introduced in December 2003 for grades 4 and 8. 
By 2005, the state plans to replace the norm-referenced exam with a criterion-referenced exam 
for grades 3-9 and 1 1 . 

New Mexico Writing Assessment Score. A one-hour writing exercise is given to students 
in grades 4 and 6 (optional in grade 8). Tests are scored on a range from 1 to 6. 

New Mexico High School Competency Exam. Students in grade 10 are tested for their 
competency in reading, language arts, math, science, social studies, and written composition. 
Results are recorded as a scale score and percentage passing for each subject category. A passing 
score is required for a high school diploma. 
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State Accountability Ranking. Each school is ranked based on test scores, number of 
dropouts, student attendance, safety plan, and parent involvement plan. Four accountability 
rankings are used by the state (exemplary, meets standards, probationary, corrective action). 
Districts are currently not ranked by the accountability system. 



Table NM7 

Student Performance Tests, New Mexico 



Test 


Type 


Scoring 


Grades 


Subject Areas 


Notes 


Terra Nova 
Survey Plus 


Norm- 

referenced 

test 


Percentile rank 
Normal curve 
equivalent 
Scale 


3-9 


Math, reading, 
language arts, 
total score, 
science, social 
studies 


Administered through 
2001-2002. Spanish Terra 
Nova was instituted 
2000-200 1 and includes two 
different tests from which 
districts could choose. 


Terra Nova 
2 nd edition 


Norm- 

referenced 

test 


Percentile rank 
Normal curve 
equivalent 
Scale 


3-9 


Math, reading, 
language arts, 
total score, 
science, social 
studies 


Replaced the Survey Plus in 
2002-2003. Scores were 
ranked using Survey Plus 
norms. 


High School 
Competency 
Exam 


Exit exam 


Pass-fail 

Scale 


10 


Reading, 
language arts, 
math, science, 
social studies 


Required for diploma. 
Spanish exit exam began in 
2000-2001. 


New Mexico 

Writing 

Assessment 


Written 

exam 


Holistic score 
from 1 to 6 


4, 6,8 
(optional) 


Writing 




New Mexico 
Criterion- 
Referenced 
Test 


Criterion- 

referenced 

test 




4,8 




The criterion-referenced test 
was piloted in two grades in 
2003-04. The state plans to 
replace the norm-referenced 
test with a criterion- 
referenced test (grades 3-9, 
11) by 2005. 



Data Usability Issues. Student performance testing in New Mexico has stayed relatively 
consistent over time and has been used for multiple grades (3-10) for all years considered in this 
study (1999-2003). These data enable researchers to track the performance of an individual 
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student from year to year. These features are highly advantageous for researchers interested in 
longitudinal analysis of resources and the performance of student cohort groups. The state’s 
commitment to including a Spanish version of their standardized test (beginning in 2000-2001) 
provides an opportunity to conduct focused analysis on limited English proficient (LEP) 
students. The most obvious concerns relate to accessibility of these data by the research 
community. State summaries of student performance are published on the New Mexico Public 
Education Department Web site, but more detailed test data are not publicly available. When 
requesting student performance data from the agency, researchers faced similar challenges 
presented in accessing Accountability Data System student and staff data (no established 
mechanism for sharing individual-level data with outside users). Changes in testing in the state 
will affect future longitudinal analysis because norm-referenced tests will be replaced by 
criterion-referenced tests in 2005. Finally, state accountability rankings provide composite scores 
for each school in the state that reflect state performance priorities. However, rankings are 
clustered in the middle two categories and provide very little variability for use in statistical 
analysis. 
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Table NM8 

Strengths and Challenges of Student Performance Data That Affect Their Use for Policy 
Research, New Mexico 



Criterion 



Strengths 



Challenges 



Availability • Student performance results in multiple 
and score formats (NCE, percentile rank, 

accessibility scale) can be requested by grade level 
from the New Mexico Public Education 
Department. 



Completeness • Student performance data can be 

requested on all test takers and also on 
subgroups (high-poverty, high-minority, 
special education, limited English 
proficient). 

• Test scores are available for a wide span 

of grades (3-9 Terra Nova), 

Accuracy 



Consistency • Terra Nova testing provides a consistent 
span of test years and grades tested. 



Alignment • Demographic and programmatic 

information about test takers can be 
matched with test results. 



• Agency staff have no procedure in place to 
provide outside users with individual- or 
grade-level student performance data. 

• Student performance data are not 
downloadable from the New Mexico 
Public Education Department Web site. 

• The universe of test takers has expanded 
with the introduction of the Spanish Terra 
Nova in 2000-200 1 . 



• Demographic data attached to student 

performance scores are unreliable due to 
self-report error; researchers must request 
additional data cross-referenced with the 
Accountability Data System. 

• The testing changed from Terra Nova 
Survey Plus to Terra Nova 2 nd edition in 
2002 (norming standard did not change). 

• New Mexico criterion-referenced test was 

piloted in 2003 and is due to replace the 
Terra Nova by 2005. 



Student, School, and District Characteristics 

Student Characteristics. New Mexico school districts compile information on individual 
students and report these data to the state via the Accountability Data System. Similar to staff 
data, student data are reported on the 40th, 80th, and 120th days of the school year. Additional 
reporting dates for students are December 1 and the end of the school year. Student 
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characteristics that are contained in the Accountability Data System and of interest for this study 
are listed in Table NM9. 



Table NM9 

Data Available in State Education Databases on Student Characteristics, New Mexico 



Student 

Characteristic 


Variables Available 


Source 


Unit of 
Analysis 


Socioeconomic 

status 


Free and reduced-price lunch program 
participation 


Accountability Data System 


Individual 


Race/ethnicity 


Ethnicity 

Tribe 


Accountability Data System 


Individual 


Grade 


Grade 


Accountability Data System 


Individual 


Gender 


Gender 


Accountability Data System 


Individual 


Program 

participation 


Special education 

Bilingual/English as a second language 
Limited English proficient/English 
language learner 
Title IA, IC, III, VIII, VII 
Johnson-O’Malley Act 
Perkins tech-prep 


Accountability Data System 


Individual 


Other 


Home language 


Accountability Data System 


Individual 



School and District Characteristics. Student characteristics can be aggregated to the 
school and district levels, creating important demographic characteristics. These measures can 
also be requested from the New Mexico Public Education Department in lieu of individual-level 
data. School and district characteristics of interest to policy researchers are listed in Table NM10 
with a short description of data sources for each. School locale code is not available in any state 
education database, and per-pupil expenditures cannot be determined at the school level because 
expenditure data are collected only at the district level. A limited number of district 
characteristics are posted to the agency’s Web site. However, these data are in portable document 
format (.pdf) and therefore are not readily usable for analysis purposes. District wealth can be 
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measured as the sum of residential, nonresidential, and oil/gas/copper values in the district, the 
district tax rate, or revenues available from local taxes (including residential, nonresidential, and 
oil/gas taxes). These district wealth measures can be downloaded from the New Mexico Public 
Education Department Web site for 2000 through 2003. 



Table NM10 

Data Sources for School and District Characteristics, New Mexico 



Characteristic 


Source 






School 


District 


School type 


Special request to the New Mexico Public 
Education Department 


N/A 


Enrollment 


Special request to the New Mexico Public Education Department or aggregate from 
student data 


Locale 


U.S. Census Bureau 2000 Census 




Attendance 

rate 


Special request to the New Mexico Public Education Department 


Dropout rate 


Special request to the New Mexico Public Education Department 


Per pupil 
expenditures 


N/A 


Calculation of expenditure and 
student enrollment data 


Average class 
size 


Special request to the New Mexico Public 
Education Department or aggregate from 
student and staff data 


New Mexico Public Education 
Department Web site or aggregate 
from student and staff data 


Number of 
teachers 


Special request to the New Mexico Public 
Education Department or aggregate from 
staff data 


New Mexico Public Education 
Department Web site or aggregate 
from staff data 


Accountability 

rating 


New Mexico Public Education Department 
Web site 


N/A 


District wealth 


N/A 


New Mexico Public Education 
Department Web site 



Community Characteristics. Education databases do not contain information about 
characteristics of the community within which a school or district is located. Census information 
provides a number of relevant indicators for community well-being, including family income, 
parent education levels, and receipt of public assistance. 
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Data Usability Issues. The quality of school and district characteristics is heavily 
dependent on the quality of their source data — student and staff data in the Accountability Data 
System (ADS) database. The New Mexico Public Education Department provides district 
characteristics on its Web site; however, only district financial characteristics can be downloaded 
for use in a research study. Also, data managers at the agency explained that the measure for 
poverty status of students is unreliable due to inconsistent reporting procedures. These data were 
only collected through the Accountability Data System during the 1998-1999 school year. A 
second source of data on students’ free and reduced-price lunch program participation are data 
that are submitted with student achievement test reports. These data are also suspect due to self- 
report error. 

Summary of Findings 

Alignment. According to data managers at the state department of education, 
characteristics of individual students and teachers could be matched by aligning students to the 
unique course number that can also be aligned to individual teachers. Student characteristics 
could also be matched to their performance scores by matching student Social Security numbers 
from the Accountability Data System and test score datasets. Although data managers described 
this process as a way to align teachers and students at an individual level, they also expressed 
that confidentiality issues would likely create a barrier for researchers to actually obtain these 
data and test whether such alignment is possible (Social Security numbers are used as unique 
identifiers for both teachers and students). 

With regard to aligning data on fiscal resources, staff salary information is collected on 
an individual level in the Accountability Data System and could be aggregated to the school and 
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district level. However, other fiscal data (e.g., expenditures for staff benefits, materials, 
equipment, travel, training) are only available at the district level. 

The certification database presents a barrier for researchers because data are continuously 
updated without annual archiving. This process, along with lack of issue dates and expiration 
dates, results in the inability to create a subset of active, certified staff for a given study period. 

Accessibility. At the time of this analysis, the New Mexico Public Education Department 
was undergoing a major restructuring. Voters approved a new governance structure in September 
2003 that eliminated existing leadership at the department. This environment contributed to 
major changes in staffing at all levels of the department; at the time of this study, approximately 
40 of the agency’s positions were unfilled. Access to data was severely limited during this time 
as few staff persons were in positions to approve outside requests for individual-level data. This 
situation created a serious barrier to the data collection process for this study. 

Usability. Based on documentation of existing data and conversations with data managers 
at the state department of education, researchers found a sufficient range of data describing 
resources; student performance; and student, school, and district characteristics. Student and 
teacher characteristics are collected by the state at the individual level, creating the potential for 
more complex analysis and the flexibility for aggregating these data to higher levels for research 
purposes. 
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Appendix E 

Texas State Education Data 

The state of Texas oversees education services to more that 4.2 million children and 
employs more than 300,000 teachers and other professional staff in 7,733 schools and 1,039 
districts. Annual education expenditures in the state top $30 billion. As one of the largest state 
education systems in the country, data collection and management have been of great importance 
to Texas and have resulted in education data resources used by state and national researchers and 
policy analysts. 

Overview of Existing State Data 

The Public Education Information Management System (PEIMS) encompasses all data 
requested and received by the Texas Education Agency about public education, including student 
demographic and academic performance, personnel, financial, and organizational information. 
According to Texas Education Agency documents, the Public Education Information 
Management System “is believed to be the world’s largest repository of educational data.” The 
Public Education Information Management System contains data necessary for the legislature 
and the Texas Education Agency to perform their legally authorized functions in overseeing 
public education. School districts submit their data to the Public Education Information 
Management System in a standardized electronic format. State education agency data managers 
also create calculated data elements derived from district submissions. Two data systems that are 
closely linked to the Public Education Information Management System are the Financial 
Accountability System and the Academic Excellence Indicator System (AEIS). 

The Financial Accountability System manages the collection and reporting of financial 
data from Texas school districts. This system helps school districts maintain budgeting and 
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financial accounting and reporting systems required by the state. It also specifies principles and 
policies that were developed by the state to ensure uniformity in accounting. Financial data are 
organized into account codes at function, object, and program levels. Data are reported through 
the Public Education Information Management System reporting process. An online resource 
guide available to the public provides guidelines for district data management staff in collecting 
and managing financial information for the Public Education Information Management System 
(http://www.tea.state.tx.us/school.finance/audit/resguidelO/index.html). The resource guide also 
contains a list and definitions of account codes (funds, functions, objects, programs) by which 
financial data are organized. 

The Academic Excellence Indicator System compiles a wide range of Public Education 
Information Management System data on students, staff, and characteristics of each school and 
district in Texas. This information is put into annual Academic Excellence Indicator System 
reports, which are available in the fall. Academic Excellence Indicator System data are also 
available for download from the Texas Education Agency’s Web site 

(http://www.tea.state.tx.us/perfreport/aeis/). These data include student performance indicators, 
student and staff demographics, staff counts, and financial summaries. The Academic Excellence 
Indicator System glossary describes each data item and provides the methodology and data 
sources for calculated values. The glossary is updated annually and can be accessed online in 
either English or Spanish at http://www.tea.state.tx.us/perfreport/aeis/2003/glossary.html. 

In addition to the Public Education Information Management System, the State Board for 
Educator Certification (SBEC) database contains information of interest to resource allocation 
researchers. The State Board for Educator Certification is responsible for ensuring that educators 
are qualified to serve in the Texas public school system. The agency issues certification 
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credentials to qualified applicants and also manages the development and administration of 
teacher competency exams. As part of their work, State Board for Educator Certification staff 
collect and report on data about Texas teachers, including certification information and teacher 
test scores. The agency’s interactive Web tool allows public users to search for information 
about teacher certification and teacher preparation institutions at 
http://www.sbec.state.tx.us/reports/default.asp. 



Table TX1 

Summary of Existing State Databases, Texas 



Data Category 


State Database 


Managing Agency/Department 


Instructional 

expenditures 


Public Education Information 
Management System (PEIMS) 
Academic Excellence Indicator 
System (AEIS) 


Texas Education Agency, School Finance 
and Fiscal Analysis Division 


Staff 

characteristics 


Public Education Information 
Management System (PEIMS) 
Academic Excellence Indicator 
System (AEIS) 


Texas Education Agency, Public Education 
Information Management System Division 


Student 

characteristics 


Public Education Information 
Management System (PEIMS) 
Academic Excellence Indicator 
System (AEIS) 


Texas Education Agency, Public Education 
Information Management System Division 


Student 

performance 


Public Education Information 
Management System (PEIMS) 
Academic Excellence Indicator 
System (AEIS) 


Texas Education Agency, Student 
Assessment Division 


Teacher 

licensure 


State Board for Educator 
Certification (SBEC) database 


State Board for Educator Certification 



Instructional Spending 

Instructional spending is tracked by school districts and organized into a standard 
accounting structure established by the Financial Accountability System. Information related to 
staff salaries is collected through the Public Education Information Management System data 
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collection for individual staff persons and integrated into the financial accounting structure. 
Descriptions of the instructional spending variables available through state education databases 
appear in Table TX2 and are discussed below. 



Table TX2 

Measures of Instructional Expenditures From District Financial Data, Texas 



Fiscal Measure 


Variables 

Available 


Description 


Unit of 
Analysis 


Instruction-related 
expenditure 
functions and 
subfunctions 


Instruction and 

instruction-related 

services 


This large function is composed of three 
subfunctions: 

1. Instruction: Expenditures for activities that 
deal directly with the interaction between 
teachers and students 


District 

School 

Program 






2. Instructional resources and media support: 
Expenditures that are directly and exclusively 
used for establishing and maintaining libraries 
and other major facilities dealing with 
educational resources and media 








3. Curriculum development and instructional 
staff development: Expenditures that are directly 
and exclusively used to aid instructional staff in 
planning, developing, and evaluating the process 
of providing learning experiences for students 






Instructional and 
school leadership 


Expenditures that relate to the managing, 
directing, supervising, and leadership of staff 
who are providing either instructional or 
instructional-related services 


District 

School 

Program 




Support 

services — student 


Expenditures that are used directly for 
noninstructional student activities or services. 
Subfunctions related to this function include 
guidance, counseling, and evaluation services; 
social work services; health services; student 
transportation; food services; and 
cocurricular/extracurricular activities. 


District 

School 

Program 




Administrative 
support services 


Expenditures that are for the purposes of 
managing or governing the school district as an 
overall entity. This function is primarily 
composed of the subfunction general 
administration. 


District 

School 

Program 
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Instruction-related 
expenditure objects 


Payroll 

costs — teachers and 


Gross salaries or wages paid to persons who 
are considered by the school district to be a 


District 

School 




other professional 
personnel 


professional staff member 


Program 




Payroll 

costs — support 
personnel 


Gross salaries or wages paid to support 
personnel 


District 

School 

Program 




Payroll 

costs — employee 
allowances 


Expenditures paid to employees for which the 
employee is not required to render a detailed 
accounting, such as contract buyouts; $ 1 ,000 
Texas Retirement System supplemental 
compensation; and automobile, housing, and 
other allowances 


District 

School 

Program 




Payroll 

costs — employee 
benefits 


Expenditures paid by the school district to 
provide benefits to employees such as federal 
Social Security payments, insurance, 
workers’ compensation, retirement, and 
unemployment compensation 


District 

School 

Program 




Professional and 


Expenditures for services rendered to the 


District 




contracted services 


school district by firms, individuals, and other 


School 






organizations, including professional 
services, tuition and transfer payments, 
education service center services, contracted 
maintenance and repair, utilities, and rentals 


Program 




Supplies and 
materials 


Expenditures for supplies and materials, 
including supplies and materials for 


District 

School 






maintenance and operations; textbooks and 
other reading materials; testing materials; and 
food service 


Program 




Other operating 
costs 


Expenditures for miscellaneous operating 
costs such as travel subsistence and stipends; 


District 

School 






insurance and bonding; elections; and 
depreciation of trust funds 


Program 




Debt service 


Expenditures for debt service including 
principal, interest, and other payments 


District 

School 

Program 




Capital 
outlay — land, 


Expenditures for capital assets, including land 
purchase and improvement; building 


District 

School 




buildings, and 
equipment 


purchase, construction, or improvement; 
furniture and equipment purchases of $5,000 
or more; and capital assets of less than $5,000 


Program 


Staff salary data 


Payroll amount 


The annual pay that a staff person is 
scheduled to receive. This amount is 
composed of a base pay for regular duties and 
supplemental pay for noninstructional 
activities such as coaching or tutoring. The 
payroll amount is associated with function, 
object, and program fiscal categories. 


Individual 
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Instructional Expenditures. Instructional expenditures are divided into four functions 
covering four broad areas of spending: (a) instruction and instruction-related services, (b) 
instructional and school leadership, (c) support services — student, and (d) administrative support 
services. These four functions are divided into sub functions that provide more detailed 
accounting of spending within the larger categories. For each function and subfunction, 
expenditures are divided into payroll costs; professional and contracted services; supplies and 
materials; other operating costs; debt service; and capital outlay — land, buildings, and 
equipment. 

Fiscal data are available at the district level and at the organizational level. Organizations 
include alternative schools, high schools, junior high/middle schools, elementary schools, and 
summer schools. Fiscal data are also organized by a range of program areas that include basic 
educational services and enhanced services (gifted and talented education, bilingual programs, 
etc.). 

Salaries and Benefits. Salary data are collected through the Public Education Information 
Management System and are reported in the following multiple formats: 

1 . The Public Education Information Management System collects individual-level 
salaries for professional, paraprofessional, and auxiliary staff organized under more than 40 role 
categories (principal, teacher, education aide, etc.). These data are the basis for aggregate salary 
data reported in the Academic Excellence Indicator System and the Financial Accountability 
System. 

2. Average actual salaries are compiled from Public Education Information 
Management System data and reported for each school through the Academic Excellence 
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Indicator System. The Academic Excellence Indicator System also reports average teacher salary 
by years of experience for each school. 

3. Salary information from the Public Education Information Management System is 
also aggregated to the school and district levels for groups of instructional staff in the Financial 
Accountability System. Salary data for related staff role categories are combined and reported as 
object-level gross salaries or wages under several function categories. Grouping of staff salaries 
allows researchers access to salary information that is aligned with fiscal function categories; 
however, the staff groupings are large enough to limit researchers’ ability to isolate the salary 
paid to any specific category of staff (e.g., classroom teachers). Under the instruction function, 
for example, the salary object represents combined expenditures to teachers, teacher aides, 
classroom assistants, graders, staff working in the classroom on a dedicated basis, adult basic 
education teachers, substitute teachers, and remote teachers. 

Two object codes in the financial data relate to staff benefits. The employee allowance 
object includes contract buyouts, a $1,000 Teacher Retirement System supplemental 
compensation, and employee allowances. The employee benefits object includes the traditional 
range of employee benefits (health insurance, Social Security, workers’ compensation, etc.). 

Data Usability. Table TX3 contains a summary of the usability characteristics that 
researchers identified regarding instructional expenditure data. Researchers found that fiscal and 
salary data are easily accessible and reliable for conducting research. The Texas Education 
Agency’s Web site contains an extensive collection of data organized by school and district. 
These data can be searched for individual schools or districts and are also easily downloadable in 
multiple computer formats. Documentation of these data is also readily available and contains 
detailed information about the data collection process and variable definitions. Salary and 
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expenditure data were collected in a relatively consistent manner during the study period 
(1999-2003), and data managers reported only minor changes to data that might affect their 
accuracy. Another important strength in this state’s expenditure and salary data is the internal 
consistency of these data. Staff salary information in the Public Education Information 
Management System and fiscal expenditure data on salaries are both recorded by function, 
object, and program and therefore represent consistent dollar amounts. These data also have the 
potential to allow researchers to align salary and expenditure data with a wide range of other data 
such as staff characteristics, teacher certification, student performance, and school and district 
characteristics. 

The Texas Education Agency uses a standardized procedure that checks for errors in data 
submitted by school districts and has a feedback mechanism that allows districts to correct 
mistakes or inconsistencies. Education Service Centers in the state use an automated system 
(EDIT+) to validate data before they allow districts to submit them to the state Public Education 
Information Management System database. Although EDIT+ can detect major errors and 
discrepancies in data prepared by school districts, it cannot identify or correct content errors or 
other less pervasive problems in data collection and reporting. Education Service Centers also 
are charged by the state to support data quality by providing training and assisting school 
districts with their data submissions. 

Researchers encountered the following challenges associated with applying instructional 
expenditure and salary data to research efforts: 

1 . Accessibility of these data is hampered by their relative complexity. A user would 
need to spend a significant amount of time learning the structure of these data and the definitions 
of variables and how they relate to one another. For example, how individual staff salaries are 
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aggregated to create the object-level salaries in the fiscal database is neither explicit in the 
documentation nor implicit in the way these data are organized. 

2. Although state data managers have a well-defined process for responding to requests 
for data that are not available via public sources, outside users must pay a fee to obtain such data 
and must wait 6-8 weeks or longer for the data request to be processed. 

3. Researchers face a challenge in computing the total amount spent for staff 
compensation because data are not available to measure the amount spent on benefits for 
individual staff. 

4. Although each individual staff salary is assigned a function, object, and program 
expenditure category for cross-referencing to the fiscal database, there is no standardized range 
of staff position codes aggregated within each function category. For example, districts could 
report the salary of an instructional officer into the fiscal expenditure function curriculum 
development and instructional staff development or into the function instructional and school 
leadership. 
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Table TX3 

Strengths and Challenges of Instructional Expenditure Data That Affect Their Use for Policy 
Research, Texas 



Criterion 



Strengths 



Challenges 



Availability • Data are available from two sources: 
and district expenditure and individual staff 

accessibility salaries. 

• School- and district-level expenditure and 
salary data are publicly available from 
the Texas Education Agency’s Web site. 

• Annually updated data documentation is 
also available on the Texas Education 

Agency’s Web site. 

Completeness • Financial data are disaggregated to the 

district, school, and program levels and 
are available for all years of the study 
period. 

• Staff salary data are collected for all 
classified and certified staff and for all 

years of the study period. 

Accuracy • The financial data collection process is 
well-established and has undergone 
minimal changes over the study period, 
increasing the accuracy of district 
reporting. 

• Editing and data validation is conducted 
by regional service centers before 
districts submit final data reports to the 

Texas Education Agency. 

Consistency • No substantive changes were made in 
data collection procedures or variable 
definitions during the study period. 

• Individual salary data are assigned 
function, object, and program labels that 

are consistent with fiscal data categories. 

Alignment • Individual staff salary data can be aligned 
with other staff characteristics. 

• Staff salary data can be averaged across 
grades, schools, and districts. 



• Data systems are large and of significant 
complexity. 

•To obtain individual staff salary data 
researchers must submit a special 
request to Texas Education Agency, 
wait for the request to be processed, and 
pay a fee. 



Some data collection problems persist, 
especially for districts that encounter 
changes in leadership or accounting 
staff. 



• Staff salaries are aggregated to function 
categories using role codes. Since role 
codes are assigned at the discretion of 
each district, there is no standard list of 
role codes aggregated into each function 
category. 

• Staff benefits are collected at the district 
level only. Individual-level expenditures 
for benefits can only be estimated by 
averaging across all staff in a district. 



Southwest Educational Development Laboratory 



152 




Investigation of Education Databases in Four States to Support Policy Research on Resource Allocation 



Staff Characteristics 

Information on staff characteristics is collected through the Public Education Information 
Management System. A wide range of staff characteristics and job-related information is 
collected for education staff. Basic demographic information and payroll information are 
collected for all staff, including nonprofessional auxiliary staff such as cafeteria workers, 
secretaries, and bus drivers. These variables include a district identification code, ethnicity, 
gender, years of experience, highest degree obtained, and payroll amount. Data on professional 
and paraprofessional staff also contain information about staff responsibilities such as the subject 
area taught, the specific position(s) held, and percentage of their full-time equivalency (FTE) 
allocated to multiple positions. These data are organized so that by identifying the role categories 
that correspond with the staff category of interest, researchers can identify a range of 
characteristics for individuals within these role categories. Classroom teachers, for example, can 
be defined with three role codes: (a) teachers, (b) special duty teachers, and (c) permanent 
substitutes. Staff in specific roles can be counted, combined, or matched to other characteristics 
such as demographics, salary, education level, or certification. 

Additional teacher information is available from the State Board of Educator 
Certification. This state agency collects certification and teacher test data on all teachers, 
including educator preparation institution, route to certification, type of certification, grade and 
subject area of certification, and raw score on teacher competency tests (pedagogy and content 
areas). This agency also compiles teacher data into reports and downloadable compilations of 
school and district data. 
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Table TX4 

Staff Characteristics Available in State Databases, Texas 



Staff 

Characteristic 


Variables Available 


Unit of 
Analysis 


Source 


Position and full- 
time equivalency 
(FTE) 


Position of staff person and full- 
time equivalency devoted to each 
position the individual fills 


Individual 


Public Education 
Information Management 
System 


Experience 


Years of experience in the 
profession 


Individual 


Public Education 
Information Management 
System 


Education 


Highest degree earned 


Individual 


Public Education 
Information Management 
System 




Bachelor’s degree institution 


Individual 


State Board for Educator 
Certification 


Demographics 


Gender 


Individual 


Public Education 
Information Management 
System 




Race/ethnicity 


Individual 


Public Education 
Information Management 
System 


Certification 

(teachers) 


Subject area of license 


Individual 


State Board for Educator 
Certification 




Grade level of license 


Individual 


State Board for Educator 
Certification 




Type of license (standard, waiver, 
emergency, etc.) 


Individual 


State Board for Educator 
Certification 




Route to certification (standard, 
alternative) 


Individual 


State Board for Educator 
Certification 




Effective date of license 


Individual 


State Board for Educator 
Certification 




Raw score on teacher assessment 


Individual 


State Board for Educator 
Certification 




Special permit 


Individual 


Public Education 
Information Management 
System 



Staff Counts and Ratios. Existing staff data enable researchers to use two obvious 
methods for determining staff counts and calculating ratios such as teacheripupil, 
administrator:teacher, and administratorpupil. 
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The Academic Excellence Indicator System school- and district-level staff counts and 
student membership counts can be used to determine staff ratios. The Academic Excellence 
Indicator System contains full-time equivalency counts of school/district administrators, 
educational aides, professional support staff, teachers, and students in each school. The limitation 
of this approach is that the lowest level of aggregation possible is the school level. Also, the staff 
counts in this database are combinations of different categories of staff. For example, the school 
administrator count combines seven staff categories from principal to athletic director. 

The Public Education Information Management System data allow greater flexibility in 
calculating staff ratios. Full-time equivalency counts of particular types of staff (teachers, 
educational aides, principals, psychologists, librarians, etc.) could be aggregated from this 
information to the school and district levels. 

Understanding that a school-level calculation of the teacheripupil ratio is not an accurate 
representation of class size, data users can consider another measure from the Academic 
Excellence Indicator System data called Class Size Average. Although these data are not a true 
measure of class size and still rely on a pupihteacher ratio, they are compiled by grade for 
elementary schools and by selected subjects for secondary schools. For elementary schools, 
teacher full-time equivalency counts in each grade level and the number of students per grade are 
used to derive a class size average. For secondary schools, each unique combination of teacher 
and class time is counted as a class, and an average is determined by summing the number of 
students served and dividing that figure by a calculated count of the number of classes in a 
subject. 
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Table TX5 

Student and Staff Counts and Ratios, Texas 



Count/Ratio Measure 


Data Available 


Unit of 
Analysis 


Source 


Staff full-time 


Can be calculated from 


By position 


Academic Excellence 


equivalency counts 


existing data 


Grade 

School 

District 

Demographic 

subgroups 


Indicator System and 
Public Education 
Information 
Management System 


Class size 


Average class size 
variable computed (with 
different computations 


Grade 

School 

District 


Academic Excellence 
Indicator System 



for elementary and 
secondary grades) 



Other ratios: 


Can be calculated from 


Grade (teachers) 


Academic Excellence 


• Pupihteacher 


existing data 


School 


Indicator System and 


• Pupihadministrator 




District 


Public Education 


• Teacher: administrator 






Information 
Management System 



Data Usability. Staff and teacher data available in Texas are comprehensive and 
complex. The Public Education Information Management System collects information on all 
education employees in the state at an individual level, and these data can be used to create 
grade, school, or district averages. The Texas Education Agency also compiles staff data at these 
levels of aggregation through the Academic Excellence Indicator System database. The 
accessibility of these data is high due to the vast amounts of data on schools and districts that it 
contains in report format or for download from the Texas Education Agency. Documentation on 
data from that agency is readily available from its Web site. A strength of the Public Education 
Information Management System data is the error-checking mechanism that was described in the 
discussion on instructional expenditures. Also contributing to the consistency of these data are 
the clear and detailed reporting instructions the state provides to reporting districts. The state has 
changed or added few variables over the past 5 years. Since data are collected for individual staff 
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and relevant grade, school, or district labels are attached to these data, they provide great 
potential for aligning to student performance data, student characteristics, and school and district 
characteristics. 

The Texas staff characteristics data also present a number of challenges for their 
application to policy research. The complexity of the staff data in the Public Education 
Information Management System database is relatively high. Understanding, for example, how a 
teacher with multiple roles and teaching responsibilities would be recorded requires careful study 
of the data documentation. Additionally, data documentation on teacher certification does not 
currently exist, and researchers must rely on conversations with data managers to fully 
understand what variables are available and their definitions. The accuracy of certain staff data 
variables is suspect. Data managers have explained that districts often apply the wrong 
instructions when recording years of experience, especially for teachers that transfer between 
districts. The State Board for Educator Certification estimates the number of in-field teachers in 
Texas schools; however, this aggregated measure is not available at an individual-teacher level 
and its validity is suspect given that the state does not collect information on teachers’ degree 
major. Data on teacher characteristics required two separate requests to two separate agencies. 
School-level teacher characteristic data are available for download from the state education 
agency, and researchers can also request data from that agency to obtain individual-level data. 
Additional teacher variables housed at the State Board for Educator Certification must be 
requested from that agency. Coordinated efforts are needed in order for researchers to align data 
sets on staff from these two agencies, so both agencies use an identical scrambling algorithm for 
teachers’ Social Security numbers. Additionally, licensure data are collected using a cumulative 
process in which updates are made to teacher information without annual archives. Researchers 



Southwest Educational Development Laboratory 



157 




Investigation of Education Databases in Four States to Support Policy Research on Resource Allocation 



must rely on the certification issue date and expiration date for each individual in order to create 
a subset of active, certified staff for a given study period. 
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Table TX6 

Strengths and Challenges of Staff Data That Affect Their Use for Policy Research, Texas 



Criterion 


Strengths 


Challenges 


Availability 

and 

accessibility 


• Individual-level staff characteristics are 
collected with ability to aggregate data up 
to grade, school, or district levels. 

• Public Education Information 
Management System and Academic 
Excellence Indicator System data 
documentation are updated annually and 
available on the Texas Education 
Agency’s Web site. 


• Data systems are large and relatively 
complex. 

• To obtain individual staff data 
researchers must submit a special 
request to the Texas Education Agency, 
wait for the request to be processed, and 
pay a fee. 

• No documentation exists on data 
contained in State Board for Educator 
Certification databases. 


Completeness 


• Information is collected for all 
employees. 


• Information about auxiliary staff is 
limited to demographic and payroll 

• . • , -i r* i pp 



information; therefore, professional staff 
that also hold auxiliary positions would 
not have complete information about 
full-time equivalency of their position. 



Accuracy 


• Automatic error checking is conducted 
before data are submitted to the state. 


• Accuracy of certain variables of interest 
are suspect (e.g., staff years of 
experience). 


Consistency 


• Few substantive changes were made in 
Public Education Information 
Management System data collection 
procedures or variable definitions during 
the study period. 

• Public Education Information 
Management System data collection 
documentation provide clear instructions 
with helpful examples for the user. 


• Teacher test scores have changed over 
time and were not required prior to 
1986. 


Alignment 


• Staff characteristics (including salaries) 
can be aligned with teacher certification 
at the individual level and student 
performance and characteristics at the 
grade, school, or district levels. 


• Staff characteristics from the Public 
Education Information Management 
System and teacher certification data 
from the State Board for Educator 
Certification are housed in different 
agencies, and researchers need to 
coordinate between them to get aligned 
data on teachers. 

• The cumulative process used to keep the 
licensure data updated create the 
potential for misaligned data. 
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Student Performance Data 

Criterion-Referenced Tests. Although the state of Texas has had statewide testing of its 
students since 1979, the performance tests most relevant for current research purposes are the 
Texas Assessment of Academic Skills (TAAS) and the Texas Assessment of Knowledge and 
Skills (TAKS). 

The Texas Assessment of Academic Skills was instituted in 1990 and has undergone 
some expansion and modification since that year. Most significantly for consideration by 
researchers is that in 1993-1994 the state began testing grades 3-9 (from 3, 5, 7, and 9) and 
moved the exit-level test from grade 1 1 to grade 10. Also during this time, the state 
accountability system began rating schools and districts, releasing tests results to the public, and 
requiring passage of the exit-level test for graduation. New passing standards were instituted in 
1994 that allowed a new measure, the Texas Learning Index (TLI), to compare achievement 
across grades. The Texas Assessment of Academic Skills was retired after the 2002 
administration and replaced with the Texas Assessment of Knowledge and Skills. Texas 
Assessment of Knowledge and Skills is a criterion-referenced exam like the Texas Assessment 
of Academic Skills but was developed to align to new performance standards (Texas Essential 
Knowledge and Skills). Texas Assessment of Knowledge and Skills tests are administered to 
grades 3-11, and results will be used to determine retention of students in grades 3, 5, and 8 and 
graduation of students at the grade 1 1 administration. 

Student test results are reported in a variety of formats (percentage passing, scale score, 
Texas Learning Index score). The Texas Education Agency also provides guidelines for 
conversions such as Texas Learning Index score to percentile rank or normal curve equivalent. 
School- and district-level test results are readily available and downloadable from the Texas 



Southwest Educational Development Laboratory 



160 




Investigation of Education Databases in Four States to Support Policy Research on Resource Allocation 



Education Agency’s Web site through the Academic Excellence Indicator System. Researchers 
can also request individual-level test scores from the agency. The Texas Education Agency will 
release individual-level data stripped of identifiers and excluding subgroups smaller than five. 

State Accountability Ranking. State statute requires annual district performance ratings 
with the standard accountability labels of exemplary, recognized, academically acceptable, and 
academically unacceptable. These labels have been determined primarily by student performance 
on state tests and dropout rates. Additional criteria will be added for determining the 2004 rating. 

Relevant for use of these rankings by researchers is the fact that since 2 years of test data 
are used to calculate the improvement portion of the ranking and the state transitioned from the 
Texas Assessment of Academic Skills to the Texas Assessment of Knowledge and Skills in 
2002-2003, rankings for 2003-2004 will not be determined, but the state will carry forward the 
rankings determined for 2002-2003. 
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Table TX7 

Student Performance Tests, Texas 



Test 


Type 


Scoring 


Grades 


Subject 

Areas 


Notes 


Texas 

Assessment 
of Academic 
Skills 


Criterion- 

referenced 


Raw score, 
scale score, 
Texas 
Learning 
Index 


3-8, 10th 
grade exit 


Math, 

reading, 

writing, 

science, 

social 

studies 


Administered in the 
spring from 1990-2002. 
A Spanish version of 
the exam was 
benchmarked beginning 
1996. 


End-of- 
course exams 


End of 
course 




Upon course 
completion 


Algebra, 
biology, 
English, U.S. 
history 


Option for meeting 
graduation requirements 
in 1995; ended 
administration in 2002 


Texas 

Assessment 

of 

Knowledge 
and Skills 


Criterion- 

referenced 




3-11 




First administered in the 
spring of 2003. 

Replaced the Texas 
Assessment of 
Academic Skills and 
end-of-course exams 



Data Usability Issues. Student performance information for Texas is available for a wide 
range of grades and over a long span of years. Statewide testing in a state as large as Texas also 
ensures that more than three million student test scores are available for research purposes each 
year. Data are reported by the Texas Education Agency in the form of Web reports and 
downloadable data down to the school level. One weakness in data usability that has arisen very 
recently is the transition from the Texas Assessment of Academic Skills to the Texas Assessment 
of Knowledge and Skills. Unless a workable conversion is made available to help equate Texas 
Assessment of Academic Skills and Texas Assessment of Knowledge and Skills scores, 
researchers are challenged to bridge the 2002-2003 and 2003-2004 test years in any longitudinal 
analysis. 
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Table TX8 

Strengths and Challenges of Student Performance Data That Affect Their Use for Policy 
Research, Texas 



Criterion 



Strengths 



Challenges 



• School and district level performance 
data (percentage passing by grade) are 
downloadable from the Texas Education 
Agency’s Web site. 

• Other score formats (Texas Learning 

Index, raw score, scale score) can be 
requested from the Texas Education 
Agency. 

Completeness • Student performance data can be 

requested on all test takers and also on 
subgroups (high poverty, high minority, 
special education, limited English 
proficient). 

• Test scores are available for a wide span 
of years (3-10 Texas Assessment of 
Academic Skills, 3-1 1 Texas 

Assessment of Knowledge and Skills). 

Accuracy 



Availability 

and 

accessibility 



Consistency • Texas Assessment of Academic Skills 
testing provides a consistent span of test 
years and grades tested until 2002. 



Alignment • Texas Assessment of Academic Skills 
reports student performance results 
using the Texas Learning Index, which 
provided researchers with better aligned 
results for longitudinal analysis. 

• Demographic and programmatic 
information about test takers can be 
matched with test results. 



• Student-level data must be obtained 
by special request from the Texas 
Education Agency; agency data 
requests may have time and cost 
implications. 



• The universe of test takers has 
expanded with the introduction of the 
Spanish and 1 1th grade versions of the 
Texas Assessment of Academic Skills. 



• Data available on the Texas Education 

Agency’s Web site is limited to 
percentage passing, limiting the full 
range of variability in test results for 
research purposes. 

• The Texas Assessment of Knowledge 
and Skills replaced the Texas 
Assessment of Academic Skills in 
2002-2003. 

• Texas Assessment of Academic Skills 

score reporting standards are not 
consistent. 

• The state has not developed a way to 
align Texas Assessment of Academic 
Skills and Texas Assessment of 
Knowledge and Skills scores for 
longitudinal comparisons. 
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Student, School, and District Characteristics 

Student Characteristics. The Public Education Information Management System database 
collects a range of individual-level student characteristics. These data are reported on the 
Academic Excellence Indicator System Web site at the grade, school, and district levels. The 
Public Education Information Management System reports on each student’s poverty status, 
race/ethnicity, date of birth, sex, home language, grade, and school. Students are also identified 
as to their eligibility for special services (migrant, bilingual, limited English proficient, at-risk, 
Title 1 Part A) and their enrollment in special programs (special education, gifted and talented 
program, English as a second language, bilingual, career and technology education, etc.). 



Table TX9 

Data Available in State Education Databases on Student Characteristics, Texas 



Student Characteristic 




Data Source 






Individual 


Grade 


School 


District 


Economically 
disadvantaged status 


Public Education 
Information 
Management System 


Academic 

Excellence 

Indicator 

System 


Academic 

Excellence 

Indicator 

System 


Academic 

Excellence 

Indicator 

System 


Race/ethnicity 


Public Education 
Information 
Management System 


Academic 

Excellence 

Indicator 

System 


Academic 

Excellence 

Indicator 

System 


Academic 

Excellence 

Indicator 

System 


Gender 


Public Education 
Information 
Management System 


Academic 

Excellence 

Indicator 

System 


Academic 

Excellence 

Indicator 

System 


Academic 

Excellence 

Indicator 

System 


Special program 
participation (special 
education, 
gifted/talented, 
bilingual/ESL, LEP, 
migrant, at-risk, Title I, 
career and technology) 


Public Education 
Information 
Management System 


Academic 

Excellence 

Indicator 

System 


Academic 

Excellence 

Indicator 

System 


Academic 

Excellence 

Indicator 

System 
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School Characteristics. The Academic Excellence Indicator System database provides 
online descriptive information about each school, including grades served (e.g., early childhood, 
K-5, 6-8), type (elementary, middle/junior high, high school), enrollment by grade, 
accountability ranking, attendance and retention rates, and student mobility rate. 

District Characteristics. In addition to the above-mentioned student and school 
characteristics that could be averaged across the district as a whole, the district characteristics 
that would be of relevance to resource allocation research includes a measure for district wealth. 
An important indicator of school district wealth in Texas is the taxable value of property. The 
Texas Comptroller of Public Accounts determines this measure on an annual basis. This 
information is available by request from the Comptroller’s office, and recent years’ values are 
compiled by the Texas Education Agency and posted on that agency’s Web site. 



Table TX10 

Data Sources for School and District Characteristics, Texas 



Characteristics 




Data Sources 




School 


District 


School type (elementary, 
high school, etc.) 


Academic Excellence 
Indicator System 


N/A 


Grade range 


Academic Excellence 
Indicator System 


Academic Excellence Indicator 
System 


Total enrollment 


Academic Excellence 
Indicator System 


Academic Excellence Indicator 
System 


Student poverty rate 


Academic Excellence 
Indicator System 


Academic Excellence Indicator 
System 


Student race/ethnicity 


Academic Excellence 
Indicator System 


Academic Excellence Indicator 
System 


Number of teachers 


Academic Excellence 
Indicator System 


Academic Excellence Indicator 
System 


Per pupil expenditure 


Academic Excellence 
Indicator System 


Academic Excellence Indicator 
System 


District wealth 


N/A 


Texas Education Agency’s Web site 


Accountability ranking 


Academic Excellence 
Indicator System 


Academic Excellence Indicator 
System 



Southwest Educational Development Laboratory 



165 




Investigation of Education Databases in Four States to Support Policy Research on Resource Allocation 



Community Characteristics. Education databases do not contain information about 
characteristics of the community within which a school and/or district is located. Census 
information provides a number of relevant indicators for community well-being, including 
family income, parent education levels, and receipt of public assistance. 

Summary of Findings 

Alignment. Data on instructional dollars; staff; teacher characteristics; student 
performance; and student, school, and district characteristics are available in Texas state 
databases. These data are housed in different data systems within the Texas Education Agency 
(Public Education Information Management System, Academic Excellence Indicator System, 
and student assessment databases) and different state agencies (State Board for Educator 
Certification, Texas Comptroller of Public Accounts). Aligning these data depends on the 
existence of common identifying variables that link one dataset to another. It also depends on the 
willingness of agencies such as the Texas Education Agency and State Board for Educator 
Certification to coordinate their responses to researchers’ data requests. A joint, cross-agency 
project is under development that addresses the need for coordinated data reporting across 
multiple data sets. The Texas Public Education Information Resource (TPEIR) database is a 
cross-agency data management system that combines primary, secondary, and higher education 
information. The Texas Education Agency, the Texas Higher Education Coordinating Board, and 
the State Board for Educator Certification are compiling data collected and managed separately 
by these agencies into one central location. Users may access both raw data and aggregated 
reports from the Texas Public Education Information Resource (TPEIR) Web site 
(http://texaseducationinfo.org/Index.asp). Although the information available currently on the 
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Web site is fairly limited, a range of important data elements are proposed or will be posted in 
the near future. 

Financial data are provided down to the school level through the Public Education 
Information Management System (PEIMS). School-level spending is organized by function, 
object, and program. As a result, researchers could identify school expenditures for instruction 
and instruction-related functions, with the information broken down by object categories. 
Researchers could also identify the amount a school spends for six specific instructional program 
areas (bilingual/ESL, career and technology, compensatory, gifted and talented, regular, and 
special education). This provides the researcher with the flexibility of aligning school spending 
to all other types of data since all that is needed is the common identifying variable of school 
identifier. Financial data are not, however, reported at the classroom or individual level in Texas. 

Information on staffing patterns such as counts, full-time equivalency counts, and ratios 
is available from the Academic Excellence Indicator System ( AEIS) Web site at the school level. 
Researchers are also able to obtain individual-level staff data that could be aggregated to a grade 
level (for teachers), school level (school staff) or district level (central office staff). Teacher 
characteristics are available on individual-level teachers and could be aggregated to grade, 
school, or district levels. These staff data could be aligned to fiscal data at the school level and to 
student data at the grade level. Although the Public Education Information Management System 
dataset contains both individual student and staff information, there is no way to link students to 
the teachers that teach their classes, hence the need to use the grade level or school indicator as 
the common identifying variable between students and teachers. The grade level a teacher is 
assigned to, however, cannot consistently be determined, and often a single teacher will have 
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assignments at multiple grades. This is especially true for teachers at the middle and high school 
level, where teachers are linked to subject areas more often than grade. 

An important consideration regarding data alignment is the fact that teacher 
characteristics such as courses taught and full-time equivalency of position are collected by the 
Texas Education Agency, while the State Board for Educator Certification collects certification 
data. Both of these agencies use the teachers’ Social Security number as a unique identifier; 
however, neither agency can release these numbers due to confidentiality policies. Additionally, 
certification data are collected using a cumulative process in which updates are made to teacher 
information without annual archives. Researchers must rely on the issue date and expiration date 
for each individual in order to create a subset of active, certified staff for a given study period. 
Data users must coordinate between the two agencies so that both use the same algorithm for 
scrambling Social Security numbers and pull the same teachers for the same study period. This 
requires significant coordination of effort, and the wait time for filling data requests at each 
agency can vary considerably. 

Accessibility. A strong benefit to using existing Texas state databases is their 
accessibility. The following two elements increase the accessibility of data: 

1 . A significant amount of school- and district-level data are already compiled into 
reports or downloadable from the Texas Education Agency’s Web site. Detailed documentation 
about data elements is also readily available from the Web site. 

2. The Texas Education Agency has public information staff persons who are charged 
with helping outside users with special data requests. The agency has standardized procedures 
for receiving and managing outside requests for data, and their confidentiality policies are clear 
and consistent. One part of the data request process that limits accessibility to data is that these 
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services are provided for a fee, and the user must budget the cost of obtaining data into the total 
cost of conducting new research. 

Usability. Overall, state education data in Texas provide a valuable and vast resource for 
education research. Databases contain a wide range of data elements available on finances, staff, 
students, schools, and districts. 

These data are not without problems, however, and according to interviews with state 
data managers in Texas, the usability of the education data they compile is highly dependent on 
the ability of local school districts to collect and report accurate and reliable information. For 
example, the years of professional experience reported for teachers are often suspect. State data 
managers say that districts often misinterpret the instructions regarding this data element and 
report the number of years a teacher has been in their district rather than in the profession. 

The relative complexity of these data also reduces their usability. Data users must rely on 
close examination of data documentation, periodic contact with state data contacts, and careful 
manipulation of the data for research use. Staff data recorded in the Public Education 
Information Management System can be particularly complex because a single staff person is 
likely to have multiple records linked by a unique identifier. Each record for a particular staff 
person will contain critical information about the individual’s position(s), classes taught, salary, 
and other information. Merging multiple staff records into a single profile for a staff person 
requires in-depth understanding of the variable definitions and data structures. 

Finally, of critical concern to researchers interested in examining the relationship 
between resources and student performance is the need for consistent measures for student 
performance. Texas has, for a number of years, expanded the number of students involved in 
testing while maintaining relative consistency in test standards through the Texas Assessment of 
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Academic Skills. With the elimination of this test and replacement with the Texas Assessment of 
Knowledge and Skills, researchers interested in longitudinal effects must either identify a 
method for aligning scores from these tests or use historical Texas Assessment of Academic 
Skills results. 
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